INSPECTOR  GENERAL 

DEPARTMENT  OF  DEFENSE 
4800  MARK  CENTER  DRIVE 
ALEXANDRIA,  VIRGINIA  22350-1500 


July  18,  2017 

Mr.  Patrick  Eddington 
Cato  Institute 

100  Massachusetts  Avenue,  NW 
Washington,  DC  20001 

Dear  Mr.  Eddington: 

This  is  in  response  to  your  Freedom  of  Information  Act  (FOIA)  request,  and  the  related  case 
pending  in  the  United  States  District  Court  for  the  District  of  Columbia,  Eddington  v.  Office  of  Inspector 
General,  Department  of  Defense,  Civil  Action  No.  l:17-cv-00128,  and  is  provided  in  compliance  with 
the  third  scheduled  production  agreed  upon  the  parties,  and  approved  by  the  court. 

Searches  of  records  maintained  by  the  Department  of  Defense  (DoD)  Office  of  Inspector  General 
identified  records  responsive  to  your  request,  and  those  records,  when  appropriate,  were  also  reviewed 
by  the  National  Security  Agency  for  consultation  on  their  equities  and  release  determination. 

One  hundred  and  sixty-two  pages  of  documents  are  being  provided  to  you  at  this  time.  Of  these 
162  pages,  19  pages,  Bates  000247-248,  000261,  000263-000269,  000271-272,  000275-276,  000283- 
284,  000307,  000309,  and  000346,  are  exempt  from  release  in  their  entirety  pursuant  to  5  U.S.C.  § 
552(b)(1),  which  pertains  to  properly  classified  information;  and  5  U.S.C.  §  552  (b)(3),  which  pertains  to 
information  exempted  from  release  by  statute,  in  this  instance  50  U.S.C.  §  3605  and/or  50  U.S.C.  § 
3024(1). 

The  remaining  143  pages  are  appropriate  for  release  with  redactions,  if  any,  at  this  time.  The 
redacted  information  is  exempt  from  release  under  the  FOIA,  specifically  FOIA  exemption  (b)(1),  which 
pertains  to  properly  classified  information;  (b)(3),  which  pertains  to  information  specifically  exempted 
from  disclosure  by  statute,  in  this  instance  50  U.S.C  §3605  and/or  50  U.S.C.  §  3024(i);  exemption 
(b)(6),  which  pertains  to  information  which  constitutes  a  clearly  unwarranted  invasion  of  an  individual’s 
personal  privacy,  in  this  instance  the  names,  phone  numbers  and  email  addresses  of  personnel;  and 
exemption  (b)(7)(C),  which  protects  personal  information  in  law  enforcement  records  which  could 
reasonably  be  expected  to  constitute  an  unwarranted  invasion  of  privacy. 


Sincerely, 

6WJJb. 

Catherine  M.  DelPrete 
Supervisory  Attorney 
FOIA,  Privacy  and  Civil  Liberties  Office 


Enclosures: 
As  stated 
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Suggestion*  for  Audits  (l)| 


(bX7)(C) 


fU)  To  suggest  ideas  for  or  to  request  future  audits  or  evaluations  of  Defense 
intelligence  issues,  contact  the  Office  of  the  Deputy  Inspector  General  for 
Intelligence  ai  i  703)  «Mjggg]Wgjg>  I  or  fax  <703 » 604-0047*  Ideal  and 
requests  can  also  be  mailed  to 


<U| 


Office  of  the  Depuiy  Inspector  General  for  Intelligence 
Attn.  Executive  Officer 

Inspector  General  of  the  Department  ot  Defense 
400  Army  Navy  DnvefeO  010  | 

Arlington.  VA  22W^Tm - 


(U) 


Defense  Hotline  (10 

(U)  To  report  fraud,  waste,  or  abuse,  contact  the  Defense  Hotline  by  calling 
(800)  424-0008,  by  sending  an  electronic  message  to  HollincaftdodiE.osd.mil:  or 
writing  the  Defense  Hotline.  The  Pentagon,  Washington,  DC  20301  *1000  The 
identity  of  each  writer  and  caller  is  fully  protected 


Acronym  (t) 

(lf)  NSA'CSS  National  Security  Agency/C  entral  Security  Service 
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INSPECTOR  GENERAL 

DEPARTMENT  OF  DEFENSE 
400  ARMY  NAVY  DRIVE 
ARLINGTON  VIRGINIA  ???0?.?A8d 


December  15.  2004 

MEMORANDUM  FOR  DIRECTOR.  NATIONAL  SECURITY  AGENCY  /  CHIEF. 

CENTRAL  SECURITY  SERVICE  (U) 

SUHJEC I  ’  Audit  of  the  Requirement  ior  the  LKA1L.BI.A/LK  and  I H IN  THREAD 
Systems  (Rcpon  No  0S-INTE1.-03)  (Project  n«.  D300MI  -0100)  (U^TOUB) 


(U)  We  are  providing  this  repun  for  review  and  comment  This  audit  was 
initiated  from  a  Hotline  complaint  We  considered  management  comments  on  a  draft  of 
this  repon  in  preparing  the  final  rcpon  Management  conducted  a  security  classification 
review  of  the  draft  repon 

(Ul  DoD  Directive  7650  3  requires  that  all  recommendations  be  resolved 
promptly.  Management  nonconcured  with  Recommendations  A. I.  and  A.2. 
Management  partially  concurred  with  Recommendations  A.3..  B.I..  B.2.,  and  B.3  As  a 
result  of  management  comments,  we  revised  Recommendations  A  2  and  B  ).  We  request 
that  the  Director.  National  Secunty  Agcncy/Central  Security  Service  provide  additional 
comments  on  all  recommendations  by  January  1 5,  2005. 

(II)  We  appreciate  the  courtesies  »-»irr>Orrl  to  the  staff  Questions  should  be 
directed  to^™^^LJal  (703)  lal 

(410)  854-PffQ'GISee  Appendix  K  for  the  RSpBrT dittrtbution  I W  l«uh  members  are 
listed  lnsKBJKtne)  back  cover 


DATED  2!  Nowvte  2004 

DECLASSE  ON  MM  I  m 


Ihomas  F  Gimble 
Deputy  Inspectof  <  tencrai  fur 
Intelligence 
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Office  of  the  Inspector  General  of  the  Department  of  Defense 

Report  No.  05-INTEL-03  December  IS.  2005 

(Project  No  D2003AL-S100) 

Requirements  for  the  TRA1LBLAZER  and  THINTHREAD 

Systems  (If) 
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Executive  Summary  (IJ) 


(U)  Who  Should  Read  This  Report  aad  Why?  Acquisition  decision  makers  at  the 
National  Security  Agency  (NSA);  the  designated  NSA  Milestone  Decision  Authority  for 
the  Under  Secretary  of  Defense  for  Acquisition.  Technology,  and  Logistics;  and 
personnel  in  the  Office  of  the  Assistant  Secretary  of  Defense  (Networks  and  Information 
Integration)  should  read  this  report  because  it  discusses  a  better  technological  and  more 
affordable  solution  for  digital  network  exploitation  than  the  solution  being  developed  by 
the  NSA 


(U//TOIJQ)  Background.  We  performed  this  audit  in  response  to  a  Defense  Hotline 
complaint,  which  alleged  that  NSA  actions  in  the  development  of  TH [NTH  READ  and 
TRAJLBLAZER  resulted  in  fraud,  waste,  and  abuse  The  specific  allegations  were:  ( I ) 
TRA1LBLAZER  development  wasted  hundreds  of  millions  of  dollars.  (2)  NSA 
disregarded  solutions  to  urgent  U.S  national  security  needs.  (3)  NSA  modified  or 
suppressed  studies  and  comparisons  that  favored  THINTHREAD  over  TRAJLBLAZER. 
and  (4)  NSA  did  not  address  FY  2002  congressional  requirements  to  deploy  and  further 
develop  THINTHREAD 


WUjJHINTHREAD  THINTHREAD  was  developed  as  a  traffic  analysis  and 
discovery  tool  to  streamline  and  improve  computer-based  exploitation  of  digital  network 
uimmmiitanun.v  < 
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_ _ _  _  ffiprcywnupi 

2000  and  has  been  deployed  loj  jadditional  sites.  However,  NSA  management  does 
not  encourage  analysts  to  use  IhINtHREAD  See  Appendix  E  for  further  details. 


(U//FTHKI)  TRAfLBLAZER.  TRAJLBLAZER  is  an  Acquisition  Category-lA  program 
to  acquire,  through  a  senes  of  spiral  developments,  an  integrated  a  vsiem  to  orwide  the 
much-needed  mission  capability  against  the  Global  Network.  |,  | 

,  [TRAILbLAZER  is  being 

developec  Sec  Appendix  U  for  further  details. 
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ftSJtaaiifta  NSA  enhanced  mating  digital  network  exploitation  systems,  under  the 
I  .‘  'i'  i  *to  Prov'<*c  *n  solution  for  digital  network 

exploitation,  wher  IHNTllRf  AD,  *  less  costly  and  more  capable  .solution,  was  already 
operational  available,  and  ready  for  deployment.  As  a  result,  the  National  Security 
Agency  is ’inefficiently  using  resource?  tb  develop  a  digital  network  exploitation  system 
that  is  not  capable  of  fully  exploiting  the  digital  network  intelligence  available  to  analysts 
from  the  Global  Information  Network.  NSA  should  deploy  THINTHREAD  as  the 

mission  and 
the 

tended  [“"1 

i  1  •  . "'~rz3 Finally.  NSA  should  develop  a  formal  training 

class  for  analysts  to  team  how  to  effectively  use  THINTHREAD  (Finding  A). 

(ll/ifOUOj  NSA  management  excluded  THINTHREAD  and  its  technologies  as  a  viable 
long-term  solution  for  digital  network  exploitation  for  the  TRAILBLAZER  Initial 
Transformation  Activities,  Technology  Demonstration  Platform  and  Objective  Program. 
As  a  result*  the  NSA  transformation  effort  may  be  developing?  less  capable  long-term 
digital  network  exploitation  solution  that  will  take  longer  andcosi  significantly  more  to 
develop.  NSA  management  should  designate  a  team  to  assess  the  ability  of  • . 
THINTHREAD  or  its  technologies  as  a  long-term  solution  tu  meei  the  f 
I  ~1  requirements  of  the  TRAILBLAZER  -T" 

Also.  A  management  should  provide  the  results  of  this 


It  to  ulC 


TRAILBLAZER  Technology  Demonstration  Platform  Milestone  Decision  Authority  for 
use  in  making  the  Milestone  B  decision  and  include  THINTHREAD  as  an  alternate 
I  _ pvutem  in  the  TRAILBLAZER  Analysis  of  Alternatives 

required  tot  Milestone  B  (Finding  B). 

1  •  • 

(If)  Management  Comments  and  Audit  Response.  The  management  comments  form 
the  Director.  NS  A/Chief,  Central  Security  Service  contained  significant  disagreements 
with  the  report  findings.  The  Director  stated  that  the  wodd's  most  advanced  cryptologic 
experts  have  assessed  the  situation  and  we  confident  in  the  approach  NSA  has  taken  The 
Director  nclnconcuned  with  the  recommendation  to  deploy  THINTHREAD  as  the  interim 
,  ..anahiiiiv  digital  nnwryt  process  and  to  use  a  complementary  approach  with 
I  -  land  THINTHREAD  in  the  oolloctiao  sites.  The  Director 

concurred  or  partially  concurred  with  the  recommendations  to  develop  a  formal  training 
class,  designate  a  teem  to  assess  the  ability  of  THINTHREAD  as  a  long-term  solution, 
and  include  this  assessment  as  pari  of  the  TRAILBLAZER  Milestone  B  decision.  The 
Program  Executive  Officer  and  TRAILBLAZER  management  will  estimate  the 
cost/ schedule  impacts  after  they  receive  estimates  from  the  review  team  and  will  decide 
at  that  time  whether  to  concur  Three  of  the  four  partial  concurrences  depend,  upon  the 
Program  Executive  Officer  and  TRAILBLAZER  management  decision  on  whether 
THINTHREAD  or  its  technologies  can  meet  the  long-term  requirement^  | 


] 


(U)  We  revised  the  report  findings  and  recommendation*  based  on  the  Director's 
comments.  However,  we  continue  to  believe  that  THINTHREAD  and  its  capabilities  can 
better  satisfy  NSA  requirements  at  leas  cost  We  request  that  the  NSA/Chief  ,  Central 
Security  Service  provide  a  risk  mitigation  plan  describing  the  steps  be  will  take  to  assess 
THINTHREAD  We  do  not  agree  with  management  that  it  would  be  detrimental  to  the 
signals  intelligence  mission  to  enhance  the|  (services  of  THINTHREAD  as 

a 
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J  (U)  the  mtenm  digital  network  exploitation  system  We  request  that  the  Director. 

NSA/Chief.  Central  Security  Service  provide  additional  comments  to  this  report  by 

J  January  15. 2005.  See  the  Findings  section  of  the  report  for  a  discussion  of  management 

comments  and  audit  responses.  Appendix  J  for  •  detailed  audit  response  to  the 
management  comments,  and  the  management  comments  section  of  the  report  for  the 
j  complete  text  of  the  comments 
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(U)  Background 
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^>tTOotUne  Allegations.  A  tetter  to  the  Defense  Hotline  requested  that  the 
Inspector  General  of  the  Department  of  Defense  audit  and  investigate  the 
acquisition  of  the  TRAILBLAZER  and  THINTHREAD  programs.  The  complaint 
charged  the  NSA  with  fraud,  waste,  abuse,  and  a  disregard  for  a  solution  to  urgent 
security  needs  because  the  TRAILBLAZER  Program  did  oot  consider  that 
THINTHREAD  Was  already  developed  to  perform  the  same  mission  Also,  the 
NSA  ignored  or  modified  the  recommendations  of  two  technology  studies  that 
favored  the  THINTHREAD  Program  Finally,  the  NSA  did  not  deploy 
THINTHREAD  to  additional  collection  sites  as  directed  by  Congress 

(V)  National  Security  Agency  Mission  and  Planning  Guidance.  The  mission 
of  the  National  Security  Agency/Central  Security  Service  (NSA/CSS)  is  to 
provide  actionable  Signals  Intelligence  to  U.S.  decisionmakers,  from  the  national 
to  the  metical  level,  while  protecting  the  nation's  vital  information  from  attacks. 
The  urgency  of  the  NSA/  CSS  mission  is  greater  than  ever  since  the  September 
1 1  th  attacks  and  the  Global  War  on  Terrorism 

(U/TFOtHMJn  addition,  defense  planning  guidance  and  the  Director  of  Central 
Intelligence  guidance  stipulate  the  need  for  **a  globally  vigilant  intelligence 
system  that  can  provide  early  strategic  warning  of  crisis  and  detect  throats"  that  is 
supported  by  capabilities  such  as  an  “information  superiority  backbone." 

(Snsi)  TRAILBLAZER  is  a  |  lmajor  acquisition  program,  which 

started  in  FY  2000  and  is  scheduled  for  initial  operating  capability  in  FY  2009,  to 
acquire  an  integrated  system  that  is  composed  largely  of  commercial  technology 
to  provide  the  much-needed  mission  capability  against  the  global  network  through 
a  senes  of  spiral  developments  The  TRAILBLAZER  system  will  deliver 
hardware  and  software,  system  integration,  and  program  integration  services  to 
include: 
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N4SA  (bXI)  14(c),  (bX3)  50  USC  3605 

1 K Ali.Bl.AZtK  program  has  not  been  stable  since  iU  infancy  and  is  undergoing 
its  third  revision. 

THINTHREAD.  The  Signals  Intelligence  Automation  Research  Center 
started  a  computer  network  defense  project  in  1996  that  became  known  as 

_Jthia  project  eventually  led  to  THINTHREAD. 
THINTHREAD  is  a  low  cost,  low  maintenance,  high  capacity,  and  high  precision 
digital  network  exploitation  system.  It  was  developed  by  a  small  team  of 
engineers  in  the  Signals  Intelligence  Automation  Research  Center  as  a  traffic 
analysis  and  discovery  tool  Because  THINTHREAD  was  considered  to  be  a 
research  and  development  project,  tbe  developers  were  told  to  discover  a  different 
approach  to  digital  network  exploitation.  Therefore,  NSA  did  not  approve  some 
of the  concepts. oqd  resources  used  THINTHREAD  wss  initially  deployed  in 
November  2000  for  developmental  evaluation  at  one  pilot  site,  using  >  high 

has  been  used  by  some  NSA  analysts  since  November  2000.1 

(U//FOUQq.  Congressional  Interest  in  THINTHREAD.  Soon  after  wort  began 
on  TRAILBLAZER,  staff  members  from  the  House  of  Representatives  Permanent 
Select  Committee  on  Intelligence  who  were  briefed  on  THINTHREAD 
capabilities  noted  that  there  were  overlaps  between  THINTHREAD  and 
TRAILBLAZER.  They  asked  why  THINTHREAD  did  not  constitute  at  least  part 
of  die  TRAILBLAZER  effort  Tbe  House  of  Representative*  Permanent  Select 
Committee  on  Intelligence  verbally  requested  NSA  to  oonduct  s  technical  review 
to  compare  the  existing  and  planned  capabilities  of  THINTHREAD  and 
TRAILBLAZER  and  requested  that  NSA  provide  tbe  committee  with  the  results 
NSA  did  not  complete  the  verbal  request,  which  resulted  in  the  Congressional 
Directed  NSA  Modernization  Study  in  the  FY  2001  Intelligence  Authorization 
Bill  (CMS  1-25-2001). 

(U)  Policy  Governing  Signals  Intelligence  System*  Acquisition.  Guidance  for 
Signals  Intelligence  system  acquisition  if  provided  in  DoD  Directive  5000.1. 

DoD  instruction  5000  2,  and  the  Clinger  Cohen  Act  of  1 996  Specifically,  these 
regulations  provide  the  framework  for  the  acquisition  of  major  automated 
information  systems 

(U)  DoD  Directive  5000.1,  “The  Defense  Acquisition  System."  May  12, 2003. 
DoD  Directive  5000. 1  establishes  the  management  process  by  which  the  DoD 
provides  effective,  affordable,  and  timely  systems  to  tbe  users.  DoD 
Directive  5000. 1  states  that  all  acquisition  professionals  shall  continuously 
develop  and  implement  initiatives  to  streamline  and  improve  the  Defense 
Acquisition  System.  Specifically,  this  directive  states  that  military  decision 
authorities  and  program  managers  shall  examine  and,  as  appropriate,  adopt 
innovative  practices  that  reduce  cycle  time  and  cost  and  encourage  teamwork 
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(U)  DoD  Instruction  5000.2,  "Operation  of  the  Defense  Acquisition  System,** 
May  12, 2003.  DoD  Instruction  5000.2  establishes  a  management  framework  for 
translating  mission  requirements  into  stable,  affordable,  and  well  managed 
acquisition  programs  that  include  weapon  systems  and  automated  information 
systems,  and  highly  sensitive  classified,  cryptologic,  and  intelligence  projects  and 
programs.  Instruction  5000.2  states  that,  during  the  initial  concept  design,  DoD 
Components  should  examine  multiple  concepts  and  material  approaches  to 
optimize  the  way  that  DoD  provides  these  capabilities.  That  approach  shall 
include  robust  analyses  that  consider  affordability,  technology  maturity,  and 
responsiveness. 

(II)  Clinger-Cohea  (Information  Technology  Management  Reform)  Act  of 
1996.  This  act  requires  that  heads  of  executive  agencies  develop  and  use  best 
practices  in  the  acquisition  of  information  technology  The  process  of  an 
executive  agency  shall  include  minimum  entena  to  be  applied  in  considering 
whether  to  undertake  a  particular  investment  in  information  systems,  including 
entena  related  to  the  quantitatively  expressed  projected  net,  risk -adjusted  return 
on  investment  and  specific  quantitative  and  qualitanve  entena  for  comparing  and 
prioritizing  alternative  information  systems  investment  projects 

(V)  NS  A/CSS  Circular  5000 R,  "Acquisition  Management,”  January  9, 2001. 
The  circular  implements  the  principles  of  DoD  Directive  5000. 1  and  DoD 
Regulation  5000.2-R  within  the  NS  A/CSS  It  establishes  baric  objectives, 
policies,  procedures,  and  responsibilities  for  acquisition  management  The 
circular  states  that  for  acquisition  management  within  the  Defense  Cryptologic 
Program.  NSA  should  address  interoperability  between  existing  and  future 
Service  tactical  Signals  Intelligence  systems,  connectivity  between  tactical  and 
national  systems,  consider  applicable  technology  that  may  satisfy  the  requirement, 
and  possible  duplication  of  ongoing  effort. 

(U)  Objectives 

(U/7FOWO)  The  overall  audit  objective  was  to  evaluate  whether  TH0MTHREAD 
should  have  been  considered  the  beat  source  for  digital  network  exploitation.  We 
did  not  review  the  management  control  program.  See  Appendix  A  for  a 
discussion  of  the  scope  and  methodology 
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A.  The  NS  A  Interim  Solution  to  Urgent 
National  Security  Needs  (U) 

Thc  NSA  deployed  enhanced  legacy  systems  and  services  under 
tbe|  |  although  requirements  could  have 

bean  satisfied  better  with  an  existing  and  more  capable  and  leas  costly 
system.  THIN  THREAD  Those  conditions  occurred  because  NS  A: 

•  (L)//ffHJQ)  did  not  consider  THINTHREAD  as  an  alternative 
solution  to  a  quick  reaction  capability  requirement  for  an  interim 
digital  network  exploitation  system  after  the  September  1 1,  2001, 
terrorist  attacks:  and 

•  (U//FPH4Q)  did  not  follow  the  recommendations  of  internal  and 
external  technical  reviews  completed  in  2001  that  recommended: 

1 .  (U//PQIX))  deploying  THINTHREAD  as  the  interim 
solution  tor  digital  network  exploitation;  and 

2.  (U/WW>)  extending  and  enhancing  the  capabilities  of 
THINTHREAD; 

I  • 

•  famigj)  delayed  deployment  of  THINTHREAD  to  collection  sites 
as  directed  by  C ongress  in  FY  2002;  and 

N)  As  a  result,  NSA  initiated  and  used  I .  ks  an  interim 

■inni*!  itfinnA  <-«r>lr>ii«ri.»n  solution,  which  NSA  plans  to  use  until  FY  2009. 

I  |  is  not  capable  of  fully  exploiting  the  digital  network 

nlelligcncc  available  to  analysts  from  the  global  information  network  and  costs 
significantly  more  to  develop,  field,  and  maintain  than  THINTHREAD 

(U)  Urgent  National  Security  Needs 


MSA  (b)(3)  50  USC  3605 


(IS*ik  NSA  documented  problems  with  the  existing  NS  A/CSS  Signals 
Intelligence  systems.  In  particular,  the  NSACSS  intelligence  system  is  oot 
capable  of  handling  the  volume  of  communications  on  the  Global  Network,  and 
(be  amount  of  intdligci.cc  dial  is  transferred  to  and  stored  in  NSA  databases  is 
unmanageable  The  NSA/CSS  Signals  Intelligence  systems,  developed  in 
response  to  the  Cold  War,  were  proficient  at  intercepting,  processing,  and 
producing  intelligence  from  analog  signals  and.  to  some  extent,  digital 
communications  on  predominantly  radio  frequency  earners.  While  analog 
communications  matured  and  spread  at  a  rate  that  signals  intelligence  technology 
was  able  to  match,  digital  technology  virtually  exploded  in  the  communications 
environment 


- - - imr 

volume  of  commumcaboas  worldwide  is  growing  rapidly,  ine  vanetv  oi 
communications  modes  and  media  are  practically  innumerable,  and  the  velocity  at 
which  the  changes  are  occurring  left  the  Signals  Intelligence  system  far  behind 


TOP  9BCRETWOQMlNlfWZOt9l  l» 


Bates  000219 


IV..  II »  f.S*N*»*W 


'J 


MSA  (b)(1)  1.4(c),  (b)(3)  50  USC  3605,  (b)(3) 
>0  USC  3024  (i) 


top  9fccnrT/eoMi*T*aei9i  m 


Bates  000220 


1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 


Bates  000221 


n.«  ii > 


i 

i 

i 

i 

i 

i 

1 

i 

i 

i 

i 

i 


N5A  M3)  56  USC  3605 


TOP  SECRETWX>MiN1iifl0a»Ba3 


\ISA  (b)(1)  1.4(c),  (b)(3)  50  USC  3605.  (b)(3)  50  USC 
_  3024  (i) 


(U)  Congressional  Guidance 


TWMt  The  FY  2002  Authorizing  Conference  language  stated  that  the 
Intelligence  Community  is  ao  formidable  in  support  of  ongoing  mature  systems 
that  it  is  impossible  to  explore  new  and  innovative  capabilities  simply  because  the 
base  does  tvot  exist.  The  intelligence  Community  may  be  sacrificing  future 
capabilities  trying  to  meet  increasing  demands  for  yesterday's  capabilities  The 
conferees  believe  that  the  TRA1LBLAZER  I  interim  or  bridge  capability  should 
focus  on  low -cost  initiatives  that  can  implement,  quickjy  and  affordably, 
significant  efficiencies,  preferably  from  the  front-end  reception  of  data  to  the 
back-end  of  processing  and  production.  The  conferee?  were  also  concerned  with 
lack  of  discetnabie  progress  made  by  the  TRAILBLAZER  1  Program,  which  had 
shown  little  progress  for  the  funds  spent  The  conferees  believed  that  the 
TRAIL-BLAZER  I  Program  would  not  promptly  deliver  capabilities  for  the 
interim 


<U) 


*  I  • 

(U/JPOlW)  In  response  to  the  terrorist  attacks  on  the  World  Trade  Center  and  the 
Pentagon  on  September  1 1.  2001 .  and  the  ensuing  wartime  threat  mode.  NSA 
needed  a  quick  reaction  capability  for  digital  network  exploitation.  Alternative 
solutions  to  satisfy  this  requirement  included  THI.NTHRF-AD  or  enhancement  of 
existing  legacy  systems  under  thel  I 
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(U)  Alternative  Solution  for  a  Quick  Reaction  Capability 
Requirement 


MSA  (b)(3)  50  USC  3605 


(SNU)  In  September  2001 .  NSA  developed  the  operational  requirements  for  a 
quick  reaction  capability  to  meet  the CilStma  tilfCal.  IH1N 1  HR  h  AH  Ifcas - 

lWMlmi u ^|ibee  A| ^ ^ 


[ 


_ _ _  subsequent  enhancements  made  to  them  as  a  part 

of  thcTT  ~  Ideveioprncnt  nor  THINTHREAD  had  the  capability 

to  meet  all  of  the  operational  requirements  identified  by  NS  A.  The  NSA  _ 

Modernization  Study  stated  that  with  additional  funding  of  approximately  I  I 
|and|  | engineers,  TH INTHREAD  .could  be  enhanced  to  fulfil) 

most,  if  not  all  the  operational  requirements  identified.  However,  for  no 


Jcngmccrs, 

II  the  operational  requirements  identified 
documented  reason.  NSA  management  did  not  consider  THINTHREAD  as  a 
viable  corporate  solution  for  the  quick  reaction  capability  requirement 


(U//FeUQXAbility  ofl  land 

THINTHREAD  to  Meet  Quick  Reaction  Capability 
Requirements 


ttitfSeveraJ  studies  identified  an  overlap  among  the  THINTHREAD  and 
TRAlLBLAZER  programs  In  March  2002,  the  THINTHREAD  Program  was 
moved  from  the  Signals  Intelligence  Automation  Research  Center  to  the 
Directorate  tor  Data  Acquisition-Cryptanalysis  and  Exploitation  Services' 
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MSA  (b)(1)  1.4(c),  (b)(3)  50  USC  3605,  (bX3)  50  USC 
i024  0) 

(iJTTPWMtThc  THINTHREAD  system  had  most  of  the  capabilities  shown  in 
T able  I  before  September  2001 .  However,  due  to  the  lack  of  management 
support  and  the  decision  to  enhance  legacy  systems,  the  original  THrNTTHRi-.AD 
development  team  was  dispersed,  and  thej  j  development 

continued  THINTHRF  AD  could  have  provided  alugBy  effective,  low-cost 
solution  for  the  quick  reaction  capability  requirement. 


9 

TOP  SECRETWCOMINW/BOW I  M3 


sISA  (bK3)  50  USC  3605 


Bates  000224 


Ilo.  il) 

MSA  (b)(1)  1.4(c),  (b)(3)  50  USC  3605,  (b)(3)  50  USC 
3024  (i) 


MSA  (b)(3)  50  USC  3605 


TOP  aEORET/J'COMlWT^O^I  113 


ill/TVlNOV-Maior  Differences  Beb» 


een  TU1NTUREAD  and 


(U/7FTO***  Flexibility  and  Modularity.  In  Jane  2002.  the  TFi  INTHREAD 
Project  Manager  stated  that  the  THINTHREAD  system  was  very  flexible, 
scalable,  and  adaptable  to  a  wide  range  of  data  ingest  environments. 
THINTHREAD  code  was  designed  to  be  reusable  and  could  quickly  be 
programmed  to  process  new  signal  types.  THINTHREAD  used  commercial -off- 
the-shelf  hardware  products  listed  on  the  NSA  Enterprise  Solutions  baseline. 
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(U//FQIIfljLTechnical  Review  Studies.  A  number  of  internal  and  external  technical 
review  studies  were  conducted  to  determine  whether  a  duplication  of  effort  existed 
between  applications  within  the  THfNTHREAD  and  TRA1LBLAZER  programs.  Those 
technical  review  studies  concentrated  on  thcl 
those  two  systems,  not  the  foil  capabilities^'  oil  sjjiuii. 

details  on  technical  reviews  conducted  on  the  THfNTHREAD  and  TRAILBLAZER 
grams 
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TSfThej  |  System  Review  end  the  THINTHREAD 

Technical  Review,  lit  early  2(J0l ,  at  the  request  of  the  NSA  Deputy  director,  • 
senior  system  engineer  was  tasked  to  conduct  a  technical  review  '  ‘ 

Buaiians«  wtf  dlffgmc*?  betw^  ryfflffn  abilities  or th  . 

ISystem,  a  subsystem  of 
‘  i  KAIlBLAZbK.  and  Ihcf  1  Hof  THINTHREAD.  The 

senior  system  engineer  personally  selected  a  panel  of  NSA  experts  to  conduct  this 
review.  The  technical  review  panel  waa  not  tasked  to  identify  the  similarities  and 
differences  betweenthe  TRAILBLAZER  and  THINTHREAD  system  capabilities 
in  their  entirety.  Anas  assessed  were  performance,  resources,  maintainability, 
scalability,  and  the  ability  to  meet  mission  requirements  An  initial  report  on  the 
findings  was  published  in  February  2001  In  March  2001 ,  the  NBA  Deputy 
Director  requested  that  a  follow-up  review  be  conducted  to  determine  bow  the 
TRAILBLAZER  Initial  Transformation  Activmf~~~~~  l 

subsystem  and  the  THINTHREal>|  •  | 


r 


inn  renter 


(U//POUO)  Results  of  the  Technical  Review.  The  technical  review  concluded 
that  significant  overlaps  existed  in  the  TRAILBLAZER  Subaysicnj 
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(U'/PDUQ)  I  jSystcm  and  the  THINTHREAD  Program  and  that 

the  approaches  and  Development  philosophies  differed.  Both  reviews  reached  the 
same  conclusions  and  recommended  that: 


MSA  (b)(3)  50  USC  3605 

1  li 


1 
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*.  engineers  to  THINTHREAD  to  continue  the  development  And  expand  the 
••  capabilities  of  THINTHREAD;  and 

•  ((U //fWWii  N S A  upper  management  should  support  THINTHREAD. 

(U)  External  Reviews 
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iSUHi  SSA  Modernisation  Studies.  The  Tint  “NS A  Modernization  Study,** 

July  2001 .  was  conducted  by  Ilic  National  Technology  Alliance.  National  Center 
for  Applied  Technology  after  a  thorough  review  or  ongoing  activities  within  the 
TRAILBLAZER  Program  Management  Office,  the  TH INTHREAD  Program 
Office,  and  interviews  with  key  program  personnel  The  Modernization  Study 
team  observed  that,  during  the  TRAILBLAZER’s  Concept  Development  Phase 
studies  and  before  conducting  an  analysis  of  alternatives,  the  TRAILBLAZER 
Program  Managcmcn ij>ffua  amlinuari  In  tkvaltm  an  interim  capability,  known 
as  the  The  team  concluded  that 

the  TRAILBLAZERr^"^^""^^™^™  l  could  become  a  potential 
"throwaway"  system,  depending  on  the  results  of  the  Concept  Development  Phase 
and  the  Analysis  of  Alternatives.  In  addition,  the  team  observod  that  NSA  . 

hesitated  to  deploy  THIN  THREAD,  a  low-cost,  readily  fiddablc,  interimj _ | 

|  The  NSA 

wwwnwoi  iMnuy  4 ,  * 

•  ffirtBI)  NSA  did" not  use  existing  products  (THINTHREAD)  to  their  lull 
potential:  „*•' 
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(U)  The  NSA  Modernization  Study  recommended  that: 


(U//P9M)  the  TRAILBLAZER  Phase  A  Initial  Transformation 
Activities  be  wHBIdMBi A IQ  avoid  itu»  JmI— |  of  the 
TRAILBLAZER] 


VISA  (b)(1)  14(c),  (bX3)  50  USC  3605,  (b)(3) 
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•  •  '  THfNl  HREAD  be  used  as  the  interim  capability  prototype 

(also  a  potential  throwaway  system  depending  upon  the  results  of  the 
Concept  Development  Phase  and  Analysis  of  Alternatives  -  but  at  a 
significantly  lower ^ost  than  thel  I 
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(U//P9laO^  The  National  Center  for.  Applied  Technology  issued  a  second  report, 
a  "quicMook"  follow-on  to  the  first  report,  in  December  2001 .  The  Chief, 
Signals  Intelligence  Programs  invited  the  original  study  team  to  conduct  a  follow- 
on  study  to  evaluate  progress  Within  the  TRAILBLAZER  Program  Management 
Office.  Despite  noting  some  changes  made  in  NSA  'management  and  the 
TRAILBLAZER  Program's  acquisition  strategy  that  promised  to  address  the 
conflicts  and  issues  noted  in  the  first  report,  the  study  team  found  that: 


the  development  of  tbd  "1  was  not 

stopped  .  A  number  of  technical  interface  issuds  arose  during  the  initial 


c 


not  met  its  planned  delivery 
Evaluation  Center.  NSA^^H 


Vi  uapner 


Lubsyst 

tSA  Test 


cm 

T  7CTJT7Prq»e  EISA  Test  and 
that  it  was  not  noccssary 


to  suspend  the  development  of  the  TRAILBLAZERf 
I  Subsystem  because  the  program 

was  well  managed,  on  schedule,  and  under  budget 

•  (U/ftJUy)  The  TRAILBLAZER  Program  Management  Office  did  not 
view  TH  INTHREAD  as  a  program  asset 

(9MH)  NSA  dispersed  the  original  TUI  NTH  READ  development  team  after  the 
technical  reviews.  The  THIbTTHREAD  Program  was  moved  from  the  Signals 
Intelligence  Automated  Research  Center  Office  to  the  Director  of  Analyses  and 
Production  -  Inno  visions  NSA  did  not  implement  the  recommendation  to 
provide |  kndl  lengineers  to  the  THINTHREAD  Program  to  extend 

and  enhance  THTNTHrFAD  capabilities.  See  Appendix  F  for  a  more  detailed 
discussion  on  the  NSA  Modernization  Studies. 
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(U)  User  Feedback  on  THINTHREAD 

>  ,  •*,  •••. 

*  *  * « •  "  •  « 

NSA  Aaalyst  Feedback.  A  team  of  analysts  did  opt  support  the 
developers  of  TftlNTHREAD  because  NSA  did  not  consider  THINTHREAD  as 
the  corporate  solytion  to  the  digital  network  exploitation  requirement  f  j 


MSA  <b)(1 )  1 .4(c),  (b)(3)  50  USC  3605 


| The  analysts,  however, 

were  not  allowed  to  assist  the  Thin  I  VAB^E >  developer*  After  the 
THINTHREAO  team  developed  the  taxonomies,  TH INTHREAD  did  not  initially 
generate  a  significant  amoiint  of  reportable  data.  However,  after  one  dedicated 
analyst,  frustrated  with  the  quality  and  quantity  of  the  intelligence  data  he 
received  from  conventional  means,  wrote  his  own  taxonomies  for 
TH  INTHREAD,  both  the  quality  and  quantity  of  intelligence  data  received  from 
THI  NTH  READ  dramatically  unproved.  The  analyst  received  intelligence  data 

that  he  was  not  able  to  receive  before  using  THINTHREAD.  The  analyst _ 

received  messages  in  the  same  context  as  they  were  sent.  I  1 

I  -  I 

(U/fal^D)  Intelligence  analysts  used  TH1NTHREAD  at  their  own  discretion 
because  'HT INTHREAD  was  not  considered  the  corporate  solution  for  digital 
network  exploitation  Intelligence  analysts  taught  themselves  how  to  use 
THINTHREAD  because  NSA  did  not  provide  users  with  system  training. 
Intelligence  analysts  who  used  THINTHREAD  stated  that  THINTHREAD: 

•  (U)  had  an  easier  query  function: 

•  (U)  was  a  faster  and  more  efficient  collection  capability; 


(U)  had  eaiicr  viewing  capabilities  because  it  was  Web  baaed; 
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•  (U//!*OUO>-madc  a  major  contribution  by  challenging  the  way  NSA 
prioritizes  and  forwards  traffic  to  the  NSA  workforce. 


1  (U)  A  complete  lywan  that  aotvaa  all  the  proctaung  problem*  itaelf  (t  hat  iU  own  input,  control.  Mama, 
monitoring,  user  interface*,  and  dale  fonnau  Even  though  other  eyttema  have  exactly  the  lamr  problems 
to  joivc.  Move-piped  sysienu  will  unplcmcm  thru  own  solutions  io  each. 
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(U)  Congressional  Directed  Action 


(U)  THINTHREAD  Deployment 
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(U)  Rationale  for  Delayed  Deployment  of  THINTHREAD 

(U)  05.  Signals  Intelligence  Directive  If  (Directive  It).  Directive  18  suites 
that  the  policy  of  the  U.S.  Signals  Intelligence  System  is  to  target  or  oollect  only 
foreign  communications.  The  Signals  Intelligence  System  will  not  intentionally 
collect  communications  to.  from  or  about  U.S  persons  or  persons  or  entities  in  the 
United  States  except  as  set  forth  in  the  Directive  18.  If  the  Signals  Intelligence 
System  inadvertently  collects  such  communications,  it  will  process,  retain,  and 
disseminate  them  only  in  accordance  with  Directive  1 8. 
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TS^Ji^  Signals  Intelligence  management  claimed  that  TH  INTHREAD 
deployment  was  delayed  because  the  NSA  General  Counsel  had  Directive  18 
issues  When  THINtHRE AD  was  deployed  to |  ^sa 

research  program,  the  Oftlcp  of  General  Counsel  unwed  me  usage  oi 
IR^AD  to  those  analysts  testing  the  TH  INTHREAD  system. 


ui;ui 


Program  Manager  also  slated  that  the  Office  of  General  Counsel  never  formally 
cited  TH  INTHREAD  with  noncompliance  of  Directive  18  since  it  was  deployed. 
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Documentation.  Signals  Intelligence  management  also  claimed  that 
the  lack  of  system  documentation  delayed  the  deployment  of  TH  INTHREAD 
The  THINTHREAD  system  began  as  a  prototype  when  only  limited  software 

documentation  was  required.  T 


I  .1  The  I  H1N  I'Hkl-.AP  Manager  stated  (hat  the 

documentation of THINTHREAD  software  was  completed  in  July  2003. 
Although  Signals  Intelligence  management  claimed  that  lack  of  documentation 
prevented  the  deployment. of  THINTHREAD  to  NS  A  collection  sites,  the 
contractor  program  manager  stated  that  THINTHREAD  could  have  been 
deployed  to  fulfill  a  Quick  Rcsnnn&r  Tanahiiitv  naH  THINTHREAD  was  also 
successfully  deployed  tol  fates,  and  NSA  personnel 

stated  that  made  enhancements  ic  the  THINTHREAD  system. 

Therefore,  the  lack  of  documentation  may  not  have  had  an  immediate  effect  on 
the  deployment  of  THINTHREAD  to  collection  sites. 


fPMMI  Studies.  Pnor  to  THINTHREAD  deployment,  the  Technology  Test  and 
Evaluation  Assessment,  March  2S.  2002.  advised  that  the  following  risk 
mitigation  and  preparation  actions  be  addressed: 


MSA  (b)(3)  50  USC  3605 


(IE/TOU0)  Provide  training  to  site  support  personnel,  implement  a 
THINTHREAD  systemwide  configuration  management  process,  and 
develop  a  process  by  which  site  personnel  can  identify  and  resolve 
problems  with  the  system  as  they  arise  to  minimize  the  operational  risk  of 
deploying  THINTHREAD  tol  I  sites; 


•  (U)  Generate  user,  system,  technical,  and  programmatic  documentation: 

•  (U)  Accelerate  deployment  planning;  develop  necessary  partnerships  with 
recipient!  kites  and  deployment  support  organizations;  and 

•  (U)  Acquire  sufficient  numbers  of  appropriately  skilled  personnel  to 
develop  documentation,  perform  deployment  planning,  support 
deployments  tol  kites,  automate  user  support  processes,  and  develop 
controlling  and  measuring  activities  for  multiple  deployments 


I 


The  Technology  Test  and  Evaluation  did  not  find  any  technical  masons  not 
to  deploy  THINTHREAD  The  NSA  Modernization  Study,  July  13,  2001, 
conducted  by  the  National  Technical  Alliance  stated  that  the  TRAILBLAZER 

auisition  strategy  did  not  accommodate  all  of  the  concepts,  approaches,  and 
iniques  being  pursued  within  the  TRAILBLAZER  Program.  In  particular,  the 
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National  Technical  Alliance  team  did  not  understand  the  rationale  for  not 


deploying  a  low-cost,  field-ready  ™p»Kit.»y  mfh  at  THINTHBFAn  r*p~-<«Hy 
since  THINTHREAD  had  hem  -deployed  tnl  I  before  the 

Concept  Development  Phase  studies  and  the  Analysis  of  Alternatives  were 
completed,  and  die  TRAIL-BLAZER  Objective  Program  Phase  B  and  C 
.  End-to-End  prototype  was  deployed 


(U)  Training  for  TH INTHREAD  Users 


(SoSI)  Analysts  at  NSA  were  not  adequately  trained  to  fully  operate  the 
TH  INTHREAD  system.  NSA  did  not  develop  formal  classroom  training  to 
instruct  THINTHkEAD  system  Users  how  to  set  up  and  operate  TH  INTO  READ 

to  enhance  users'  ability  to  collect,  process,  and  exploit  digital  network _ 

intelligence  finon| 

communicanormTiignTtets.  .aw'CTTPSB  taugn  manseim  munmwmam - 

provided  user  training  to  other  analysts.  The  On-the-Job  Handbook  enables 
THINTHREAfl  "jopergtions  personnel  to  leant  about  the  system  in  •  sclf- 
paoed  instructional  format.  The  goal  of  the  Handbook  is  to  lead  trainees  to  a  skill 
level  that  would  allow  them  to  effectively  operate  the  TH  INTHREAD  system 


"T9W0H  Although  many  analysts  stated  that  TO INTO  READ  is  user  friendly,  a 
formal  training  course  would  provide  users  with  a  frill  overview  of  the 
THINTHREAI)  capabilities  and  operational  procedures.  Educating  users  in  how 
to  create  simple  andl  lis  very  important.  Analysts 

must  carefully  1  * 
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**  4  '  |  | 

"because  NSA  did  not  provide  tnrmal  classroom 

tranuog.  analysts  were  reluctant  to  use  THINTHREAD. 

•  *  •  • 

•  -V  \  .  • 

(U)  Summary 

•  M  V,  »  \  /' 

*€*0fyThe  THINTHREAD  system  has  the  capability  to  provide  a  ciost-effcctivc. 
high  capacity,  high  precision  digital  network  exploitation  system  that  should  be 
immediately  deployed  to  perform  parts,  of  the  interim  digital  network  exploitation 
mission.  Studies  completed  by  internal  technical  review  teams  and  an  external 
team  of  exports  concluded  that  the  THINTHREAD  system,  its  technologies,  and 
its  approach  to  d until  network  exploitation  were  better  than  NSA*s  current 

capabilities.  1  | 

I  rncnjT!  system  aisc  scores  ~ 
nagBgHgaifd  snalysta,  and  has  a  very 

_ _ . _ _ _  l  THINTHREAD  provides 

the  systems  capability  that  NSA  need-  u>  keep  pace  with  the  Global  Information 
Environment  THINTHREAD  produces  valued,  unique  intelligence  that  meets 
customer  needs  in  a  timely,  oost -effective  manner,  that  is  driven  by  the  needs  of 
its  customers',  and  that  is  flexible,  adaptable,  and  agile  enough  to  anticipate  and 
respond  to  the  changing  needs  of  NSA,  Signals  Intelligence  targets,  and 
technologies  ; 

a  «  »  .  *B  *, 

mrrsti  Although  die  House  Permanent  Select  Committee  Intelligence 
Authorization  Act  for  FY  2001  directed  NSA  to  deploy  THINTHREAD  to 

I  _ _ Icollcction  sites  immediately.  THINTHREAD  was 

deployed  sites  Over  a  2-vear  omod.  According  to  a  2002 

briefing  to  the  House  Permanent  Select  Committee  on  Intelligence,  I  hr 

those  sites  are  considered  optimal] _ '  '  )  collection  sites.  The  NSA 

delayed  deployment  of THINTHREAD  because  Signals  Intelligence  management 
transferred  THINTHREAD  from  the  development  team.  Signals  Intelligence 
management  claimed  that  THINTHREAD  did  not  comply  with  Directive  1 8  and 
THINTHREAD  developers  did  not  document  system  development  to  allow 
deployment  Although  studies  recommended  that  NSA  deploy  THINTHREAD  as 
an  interim  solution  tod  as  a  risk-reduction  test  bed  for  the  TRAILBLAZER 
Program.  THINTHREAD  was  not  timely  deployed  as  a  complete, 
mission-critical,  digital  network  exploitation  capability  In  addition.  NSA  did  not 
develop  «  formal  training  progtam  for  TH  INTHREAD  users.  As  a  result, 

I  hnalyats  did  not  have  access  to  all  digital  network  intelligence 

data  to  assist  in  the  development  of  intelligence  reports  NSA  management 
should  consider  implementing  the  following  recommendations  before  continuing 


(U)  Management  Comments  on  the  Finding  and  Audit 
Response 

(1J/1TOUQ)  Management  Comments.  The  NSA  management  stated  that  the 
draft  report  did  not  include  significant  information  or  included  wrong  or 
misleading  information.  The  complete  list  of  management  comments  is  in  the 
Management  Comments  section  of  this  report 
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(IW/PtlttQl  Audit  Response.  We  obtained,  reviewed,  and  analyzed  many 
documents  and  we  interviewed  several  NSA  and  contractor  personnel  to  form  the 
conclusions  in  this  report.  We  included  all  information  relevant  to  support  die 
report  findings  and  recommendations.  Many  people  we  interviewed  asked  not  to 
be  identified  for  fear  of  management  reprisal.  We  gathered  information  from 
many  sources  to  support  the  finding.  Paragraphs  frequently  contain  information 
from  several  sources  to  reinforce  conclusions.  Baaed  on  management  comments, 
we  made  various  editorial  changes  to  the  draft  report  See  Appendix  J  for  a 
completed  audit  response  to  each  management  comment. 

(U)  Recommendations,  Management  Comments,  and  Audit 
Response 


Revised  Recommendation.  As  a  result  of  management  comments,  we 
revised  Recommendation  A.2  to  request  that  TH INTHREAD  be  folly  enhanced 
and  extended  to  include  handling  the  necessary  services  as  well  ad 


(U)  A.  We  recommend  that  the  Director.  National  Security  Agency/Chlef, 
Central  Security  Service: 

1.  (W9I)  Deploy  THlNTHREAD  ns  the  interim  capability  digital 
network  exploitation  system. 

'{SriSi}  Management  Comments.  The  Director,  National  Security  Agency/Ouef. 
Central  Security  Service  nonconcured  and  stated  that  the  systems  engineering 
management  integrated  THINTHREAD  concepts  into!  | 

NSA  management  stated  it  would  be  detrimental  to  remove  non-TH  INTHF  FAD 
capabilities  from  the  existing  digital  network  exploitation  architecture.  The 
National  Security  Agency  has  consolidated  processing  activities  by  improving 
cxisUnit  capabilities  and  developing  new  capabilities  to  create 
1  I  “  overarching  digital  nflWMfc  tanlOHalian  PTOmun. 

/  -.  Annex  A  ot  the  management  comments  compares^ 

services  with  THINTHREAD  services  NSA  management  stated  that 
I  "  •hnrvices  exceed  TTHNTHREaD  system  capabilities. 

TScm  Audit  Response.  We  accept  that  THINTHREAD  capabilities  are  already 
deployed  within  the  National  Security  Agency's  I  |  analysis 

and  production  programs  as  a  result  of  previous  congressional  direction,  sound 
systems, engineering  practices,  and  common  sense.  However,  even  though 
THINTHREAD  capabilities  and  some  lessons. learned  were  integrated  into  some 
components  ofl  1 3  years  after  tip  start  of  initial 

development,! 


wnp 

analysts  wefeqot 
the  systcrpS’  frill 


I  THIN  I HKEAD  systems  were  deployed 

igency  did  not  emphasize  (he  use  of  those  systems, 
trained  to  use  it,  and  it  was  not  deployed  to  sites  where 
can  be  properly  used  and  tested. 


MSA  (bX3)  50  USC  3605 
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ISWSf)  MiugeoKBt  Comments  The.  Director,  National  Security  Agency/Chief. 
Central  Security  Service  nonconcurred  because  the  National  Security  Agency 

manaomtmf  fhat  mtonihrin  frmrri  I 


ITIINTHREAD 
enhanced  and  extended 


Management  did  not  understand 
ended  THfNTHREAD”  in  the  dri 


meant  by  “fully 
the  drift  version  of  this  report. 


(Ml)  Audit  Response.  Based  on  management  comments,  we  revised  the 
recommendation  to  explain  more  clearly  the  meaning  of  “fully  enhanced  and 
extended  TH INTHREAD. "  We  recognize  that  the  National  Security 
increased  its  digital  network  exploitation  | 
over  the  past  few  years,  however,  the  existing  system,  \ 
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MSA  (b)(1)  1  4(c),  (b)(3)  50  USC  3605 

MSA  (bXI)  14(c),  (bX3)  50  USC  3605 

-fTOiMOl)  that  the  National  Sccunty  Agency  management  reconsider  its  position 
and  PTCYldC  MddiPOMl  womraents  on  using  a  complementary  approach  with 
I  hntj  THlNTHREAD  in  the  collection  sites,  where 

required  until  THlNttikEAD  is  fully  enhanced  and  extended  tt>|  | 


jJS)  Develop  formal  classroom  training  for  the  entire  - - 

THIMTflREAP  system  to  Include  the  function  of  each  component  j 

- J 

( U/TTlMJ  Of  Management  Comments.  The  Director,  National  Security 
Agencyi'Chicf,  Central  Security  Service  concurred.  In  April  2003,  NSA  provided 
350  pages  of  guided  courseware  in  Dau  Item  DI-ALSS-8I523A. 

tef  Audit  Response.  Management  comments  were  partially  responsive. 
Although  the  National  Security  Agency  integrated  THlNTHREAD  components 
into  the  Signals  Intelligence  Systran,  it  still  needs  to  develop  a  formal  training 
course  for  the  THlNTHREAD  end-to-end  system  We  request  that  management 
reconsider  its  position  towards  providing  formal  classroom  training  and  submit 
additional  comments  by  January  I S.  2005 


MSA  (b)(3)  50  use  3605 
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B.  The  NSA  Transformation  (U) 


Jfltf  NSA  management  excluded  THINTHREAD  and  its  technologies  as  a 
viable  long-term  solution  for  digital  network  exploitation  for  the 
TRAIL-BLAZER  Initial  Transformation  Activities,  Technology 
Demonstration  Platform,  and  Objective  Program.  This  condition  occurred 
because  NSA  management  did  not  consider  independent  and  internal 
analyses  recommending  that  NSA  use  THINTHREAD  and  its 
technologies  as  a  risk -reduction  test  bed  for  the  TRAILBLAZER  Program 
and  should  include  TH INTHREAD  in  the  TRAILBLAZER  Analysis  of 
Alternatives.  As  a  result,  the  NSA  j  | 

I"  -  1  _  ~  1 

f~*^~~~~That  will  eventually  be  integrated  with  the  TRAILBLAZER 
Objective  Program.  In  addition,  NSA  may  be  developing  parts  of 
TRAILBLAZER  Technology  Demonstration  Platform  unnecessarily 
Finally,  the  NSA  Transformation  may  be  developing  a  less  capable,  long¬ 
term  digital  network  exploitation  solution  that  will  take  longer  and  cost 

significantly  more  to  develop. 

• 

(U)  Congressional  Concerns  on  Digital  Network  Exploitation 


(U)  The  National  Defense  Authorization  Act  for  FY-2002  stated  that: 


4  *  , 

The  NSA  has  clearly  made  great;  strides  to  seeking  to 
transform  itself  in  many  areas  However,  the  NSA  appears  to  have 
mack  only  modest  progress  to  the  area  moat  important  to  ifc  tom: 
icqtpnng  the  technical  ability  to  operate  effectively  against  (he 


— ■ffi/ZBI)  On  March  24, 2000.  the  Dirortor,  NSA  signed  the  TRAILBLAZER  1 
charter  to  create  p  new  Signals  Intelligence  enterprise  to  exploit  the  global 
network.  TRAJUlBLAZLR  i,  with  an  estimated  development  cost  of 
approximately  |  *  L  was  to  respond  to  the  opportunities  and  challenges 

provided  by  emerging  technologies  to  design  new  operational  practices  for  on- 
time  delivery  of  products  and  services  in  the  manner  that  best  met  customer 


IJ5ATO50U5C  3605 


27 

top  seemwcoiWNmewi  m 


Bates  000242 


TOP  flEC'RETflCOMlNTrtgOtgl  123 


(IM6K  needs- unrestrained  by  past  practices  or  past  technologies. 
TRAILBLAZER  I  had  the  following  key  responsibilities 


•  (U//P0U©)1  Ensure  the  execution  of  a  new  NSA  program  transforming 
the  U.S.  Cryptologic  System  exploitation  of  the  global  network.  Map 
detailed  digital  network  exploitation  mission  requirements  to  a 
cryptologic  architecture  and  implement  them,  and 

•  (U//POU01  Act  as  the  catalyst  for  transforming  customer  interaction 
and  operational  practices  Begin  the  retooling  of  NSA 
programmatically,  technologically,  and  culturally  to  provide  mission 
services  and  capabilities  that  are  responsive  to  the  emerging 
technology  and  customer  needs. 

(U)  An  Alternative  Proposal  to  TRAILBLAZER 


W5A(bXtH.4(c>.{bX3)50U5C36fl5 


in  January  2001 .  the  TRAILBLAZER  Alternative  Proposal  identified  the 
capabilities  of  the  newly  developed  TH  INTHREAD  system,  as  well  as  future 
development  efforts  that  would  meet  most  of  foe  requirements  in  foe 
TRAILBLAZER  Statement  qf  Objectives  and  its  Addendum.  The  proposal 
UNTHREAD f 


demonstrated  that  the  TH1 


Program  had  achieved  moat  of  the 


requirements  in  foe  TRAILBLAZER  Statement  of  Objectives  during  site  testing. 
The  proposal  contended  that  THIT 
than  120  days  and  at  a  cost  otj~ 


Object] 

READ  achieved  those  requirements  in  less 


(0*0H  The  proposal  stated  that  TH  INTHREAD  was  already  processing  a 
significant  percentage  of  the  digital  network  intelligence  for  thej _ ___J 


(U//F0UQ)  Because  of  NSA  management's  perceptions  of  required  architectural 
features,  foe  proposal  faced  a  number  of  challenges  because 
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•  (U/PtW©T  The  implied  development  and  deployment  costs  of 

■  THINTJ-IRF  AD  were  dramatically  lower,  and  the  development  schedules 
to  satisfy  TRAILBLAZER  requirements  were  dramatically  shorter  than 
previously  estimated,  bringing  into  question  the  completeness  of  other 
proposals  \ 

HJMSU.NSA  did  not  implement  the  proposal.  If  it  had  been  unplememed  or  if 
the  technical  approach  and  breakthrough  concepts  developed  by  the 
THINTHREAD  team  were  used  as  a  risk  reduction  teat  bed  and  lesson  learned 
for  TRAIL-BLAZER .  a  substantial  reduction  in  TRAILBLAZER  developmem 
costs  may  have  resulted 


t  ,  »  •  • 

(U)  NSA  Modernization  Studies 

•  \ 

PCH. First  NSA  Modernization  Study,  the  House  Permanent  Select  Committee 
on  Intelligence  directed  the  first  “NSA  Modernization  Study,"  July  2001,  which 
was  conducted  by  The  National  Technology  Alliance.  National  Center  for 
Applied  Technology.  After  a  thorough  review  of  ongoing  activities  within  the 
TRAILBLAZER  Program  Management  Office  and  the  THINTHREAD  Program 
Office  and  interviews,  with  key  program  personnel,  the  National  Center  for 
Applied  Technology  ttam  pbservcd  that  the  TRAILBLAZER  was  poorly 
executed  and  had  an  overly  expcnatvc|  pystem. 

The  NSA  Modernization  Study  recommend M  mar 

•  •  • 

•  id  THINTHREAD  be  used  as  a  nsk -reduction  test  bed  and  provide 
TRAILBLAZER  with  lesson  teamed;  and 

t  •  _ . _ 

•  jg^THINTHREAD  be  considered  as  an  alternative  T 

I  lystein  in  the' TRAILBLAZER  Analysis  of  Alternative^ 

Second  NSA  Modernization  Study.  A  second  National  Center  for 
Applied  Technology  report,  a  "quick-look"  follow-on  report,  was  completed  in 
December  2001 .  The  Chief,  Signals'intelligcnce  Programs  invited  the  original 
study  team  to  conduct  a  follow  »on  study  to  evaluate  progress  within  the 
TRAILBLAZER  Program  Management  Office  Despite  noting  some  changes 
made  in  NSA  management  and  the  TRAILBLAZER  Program  acquisition  strategy 
that  promised  to  address  the  conflicts  arid  issues  noted  in  the  first  report,  the  study 
team  found  that; 


(U//lT5U*N  Signals  Intelligence  programs  did  not  deploy  THINTHREAD 
as  a  risk-reduction  test  bed  arid  provide  lessons- 1  earned  for 
TRAILBLAZER  in  addition.  Signal  Intelligence  management  had 
directed  that  THINTHREAD  resources  be  provided  to  TRAILBLAZER  to 
support  the  mi 
TRAILS  LAZE 


of  THINTHREAD  technology  into  the 
"{architecture;  and 


•  (1 1/^fH.IQ)  The  study  also  recommended  that  THINTHREAD  be 
considered  as  an  alternative  |  {system  in  the 

TRAILBLAZER  Analysis or  Alin FIHUVW 


J 
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(U'/RdUttt  The  study  also  stated  that  Signals  Intelligence  programs  would  not 
consider  THINTHREAD  as  an  alternative  framework  for  TRAILS LAZER  and 
ruled  out  TH INTHREAD  as  an  alternative  solution  because  it  was  viewed  as  a 
mission  application. 

(U)  MSA  Technical  Reviews 


ISi'i'Otf  During  FY  2001,  NSA  initial  and  followup  iniernal  technical  reviews 

idamfigi  tfygliPt  bfly  WD.thg  i  ,  L  ,  '  ,  1 

I  frubSV^Sri  arrJiltccntfe  tut  WM  rve|np  rtrvrlonwtl  »rvt 


the  I  HINTHKJ-'AD  I 


included  in  the  ^  ^  TRAlLBCAZKR ’Objective  Program  Roth  reviews 

concluded  that  tH  INTHREAD  and  its  technologies  should  be  adopted  into  the 

i  to  digital  network  exploitation.  Tha  i  | 


i  in 


l>f  programmers  The  requested  resources  for  TH  INTHREAD  were 

Sd  [jhandpickcd  programmers  to  satisfy  the  full  request  for 
e  fSCnnical  review  also  recommended  that  NSA  provide  the 
resources,  dollars,  and  people  to  the  TH  INTHREAD  team. 

(U)  TRAILBLAZER  Acquisition 

•  *  »  - .  '  *  .  , 

After  ’spcndingrWcars  and  more  than  1  Ton  <feve^°Praenl  «nd 

changes  to  the  TRA1LBT.AZER  acquisition  strategy,  NSA  was  unsuccessful  in 
delivering  a  capability  to  fully  exploit  the  global  network 


O )//fUMO)  TRAILBLAZER  Initial  Transformation  Activities  The 

objective  of  the  TRA1LBLA2ER  Initial  Transformabon  Activities  was  to  provide 
a  global  network  learning  environment  that  would  feed  information,  lessons 
learned,  new  analytic  processes,  and  refined  requirements  into  the 
TRAILBLAZER  Objective  Program  Activities 


7^4^ >unng  FY  2001.  the 

^•ht|R,|LBUAZE,Wfm|KATr 


I  Studies  identified  o 


glaps 


The  studies  concluded 
by  developing  the 
]  before  the  TRAILBLAZER 

The  overall  objective  of 


>A  was  potentially  developing  a  i 
TRAILBLAZER!  _  ' 

Concept  Definiboo~FF)!astr  5luJ\  Program  was  complete, 
the  Study  Program  was  to  identify  potential  solutions  for  the  global  network 
challenge  and  to  position  the  TRAILBLAZER  Program  for  the  next  phase  of  the 

program  acquisition.  The  decision  by  NSA  management  not  to  follow  the_ _ 

Modernization  Studies’  recommendations  to  stop  the  development  of  the  I  I 

|  |  resulted  in  an  expenditure  of  approximate! 4  Ifor 

1  A  iKmvipcmumgrpHdotypc  As  stated  in  finding  A  the  Modernization  Studies 
recommended  adopting  TH  INTHREAD  as  the  interim  solution  for  digital 
network  exploitation  and  including  TH  INTHREAD  in  the  Analysis  of 
Altemativea  for  TRAILBLAZER. 


(ll//Wbftl  Refining  the  Scope  of  the  TRAILBLAZER  Program.  The 
terrorist  attacks  against  the  United  States  on  September  1 1 . 2001.  established  an 


JO 
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<U/FQUtt|  urgent  mission  statement  for  NSA  to  rapidly  develop  and  field  a 
capability  for  digital  network  exploitation.  As  documented  in  a  memorandum 
from  the  Chief,  Operations  Requirements  (Signals  Intelligence)  to  the  NSA 
Senior  Acquisition  Executive.  “Urgency  Statement  for  Global  Capabilities.*'  dated 
November  9.  2001.  such  a  capability  was  deemed  critical  to  effectively  process, 
analyze,  and  report  information  from  the  global  network  The  cited  memorandum 
documented  two  key  decisions 

•  (6)  NSA  must  refocus  the  initial  acquisition  activities  for 

TRAILBLAZER  to  counter  terrorism;  and 


•  (Q  NSA  must  move  the  TRAILBLAZER  Program  forward  swiftly  to 
deliver  the  capability  needed  to  extract  information  from  the  global 
network  by  the  second  quarter  of  FY  2003  to  ensure  that  the  United  States 
has  the  advantage  in  the  war  on  terrorism. 

a 

(L7JptHMfc)  The  mission  statement  was  a  key  driver  in  the  formation  of  the 
Technology  Demonstration  Platform  The  Acquisition  Review  Board  approved 
the  revised  acquisition  strategy  on  November  30,  2001 . 
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(U)  Conclusion 

(9^91)  TRAILBLAZER  was  developed  to  create  a  new  Signals  Intelligence 
enterprise  to  exploit  the  global  network.  The  TRAILBLAZER  Technology 
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an  alternative 


The  studies  recommended  that  THINTHREAD  be  considered  as 
□  system  in  the  TRAIIBLAZER 


Analysis  of  Alternatives.  but  because  N^A  did  not  consider  THINTHREAD  as  a 
viable  long-term  solution,  it  did  not  include  THINTHREAD  in  the  Analysis  of 
Alternatives  As  a  good  business  practice,  the  NSA  should  reevaluate  ; 
THINTHREAD  as  the  long-term  solution  to  exploit  the  global  network,  because 
NSA  may  be  unnecessarily  developing  the  |  -feibaystem 

logy  Den 

sfol 


and  parts  of  or  all  of  1RAILB 
management  should  consider  i 
before  continuing  the  development  of  the 


Ti7Yn  rrrr 


NSA 

,  the  following  recommendations 
echnology  Demonstration  Platform 


(U)  Summary  of  Management  Comments  on  the  Audit  Finding 
and  Audit  Response 

•  $ 

(U//FOUCH  Management  Comments.  NSA  management  referred  to  several 
statements  in  the  report  that  did  not  include  all  referenced  documents  and  that 
some  paragraphs  contained  wrong  or  misleading  information  The  complete  list 
of  management  comments  is  located  in  die  Management  Comments  section  of 
this  report. 

(XJ//FUI**)  Audit  Response.  We  obtained,  reviewed,  and  analyzed  many 
documents  to  form  the  basis  for  the  conclusions  in  this  report.  We  also 
interviewed  several  NSA  and  contractor  personnel,  many  of  whom  asked  not  to 
be  identified  for  fear  of  management  reprisal  We  gathered  information  from 
many  sources  to  support  the  finding  Paragraphs  frequently  contain  information 
from  several  sources  to  reinforce  conclusions.  See  Appendix  J  for  a  complete 
audit  response  to  each  management  comment 

*  • 

(U)  Recommendations,  Management  Comments,  and  Audit 
Response 

(S)  Revised  Recommendation.  As  a  result  of  management  comments,  we 
revised  Recommendations  B  1  and  B.3  to  clarify  the  intended  use  of 
THINTHREAD  as  a|  ">tem 

1.  (1/fOUO)  We  recommend  that  the  Director,  National  Security 
Agency /Chief,  Central  Security  Service: 

•  Designate  a  teem  to  atseti  the  ability  of  THINTHREAD  or  its 

lechndopa  as  a  long-term  sotatioa  to  meet  the  |  - 1 

I  "Requirements  for  the  TR A ILBLAZEK  Technology 

Demon ifraoon  Platform 


(ll/ffOtO|  Management  Comments.  The  Director.  National  Security 
Agency/Chief.  Central  Security  Service  partially  concurred  with  the  intent  of  the 
recommendation  but  was  waiting  for  the  results  of  a  cost  and  schedule  impact 
study  before  fully  concurring.  The  Program  Executive  Office  is  establishing  an 
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(Iv/ptW©)  independent  review  team  tn  complete  the  study  before  Milestone  B. 
scheduled  for  December  2004,  because  the  study  will  have  an  effect  on  both  the 
cost  and  schedule  for  Milestone  B 


(S^Si)  Management  slated  that  there  were  significant  differences  between 
THINTHREAD  and  TRAILBI AZER.  THINTHREAD  is  a  digital  network 
information  processor  and  will  provide  capabilities  to  support  analysts'  activities 
TRAtLBLAZER  will  provide  the  enterprise  wide  set  of  digital  network 
requirements  The  information  technology  framework  will  boat  and  integrate 
mission  applications 

-*{6}  Audit  Response.  Based  on  management  comments,  we  revised  this 
recommendation.  The  Director.  National  Security  Agency/Chief,  Central 
Security  Service  comments  were  responsive  and  the  proposed  actions  meet  the 
intent  of  the  recommendation  The  immediate  actions  taken  to  designate  an 
independent  review  team  to  perform  the  assessment  are  commendable 
Milestone  B  hasslippbd  to  February  2005  at  the  earliest,  therefore.  NSA  will  have 
time  to  complete  the  assessment  of  THINTHREAD  or  its  technologies  as  a  long¬ 
term  solution  to  meet  thef  | requirements  of 

tRAlLBLAZER.  We  request  that  the  Director.  National  Security  Agency/Chief, 
Central  Security  Service  provide  a  nsk  mitigation  plan  for  cost  and  schedule 
requirements  and  provide  comments  on  a  final  concurrence. 

2.  (U//HHJO)  fi  uytde  the  results  of  the  external  assessment  to  the 
TRAILBI -AZER  Technology  Demonstration  Platform  Milestone  Decision 
Authority  for  use  in  making  the  Milestone  B  decision. 

(U//F0UQI  Management  Comments.  The  Director,  National  Security 
Agency/Chief,  Central  Security  Service  concurred.  The  completion  of  the 
independent  assessment  is  currently  scheduled  for  December  2004. 

I  t 

(L//POUUq.  Audit  Response.  Although  the  Director.  National  Security 
Agency/Chief.  Central  Security  Service  concurred,  the  concurrence  is  based  on 
completing  the  assessment  requested  in  Recommendation  B.  I  We  request  that 
(he  National  Security  Agency  management,  provide  additional  comments  in 
response  to  the  final  report  identifying  NSA  .plans  to  eliminate  the  cost  and 
schedule  risks  for  completing  the  assessment. 
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lor  Milestone  B 


■TPHnclude  THINTHREAD  as  an  alternate  | 
^system  in  the  TRAJLBLAZER  Analysis  oi  Aftcm 


SHves 


required 


(II//FTHJO)  Management  Comments.  The  Director.  National  Security 
Agency/Chief,  Centra)  Security  Service  concurred,  and  staled  that  the  assessment 
required  in  Recommendation  B.  I  will  be  included  in  the  Analysis  of  Alternatives 
for  Milestone  B.  scheduled  for  December  2004 

<6081)  Audit  Response.  Although  the  Director.  National  Security  Agency/Chief, 
Central  Security  Service  concurred,  the  concurrence  is  based  oh  completing  the 
assessment  requested  in  Recommendation  B.l.  Based  on  management  comments 
to  Recommendation  B.2.,  this  recommendation  was  revised  to  include 
THINTHREAD  as  an  alternate  digital  network  information  I  I 
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Ta^ftbj  I  system  that  TRA1LBLAZER  will  host  We  request  additional 

comments  in  response  to  the  final  report  identifying  NSA  plans  to  eliminate  the 
cost  and  schedule  risks  for  completing  the  assessment 
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Appendix  A.  Scope  and  Methodology  (U) 

•  t 
•  # 

(U  'TTOUOfWc  reviewed  documentation  daung  from  November  2000  through 
October  2003  including  background  information,  mission  and  operational  need 
statements,  concepts  and  operational  requirements,  and  contracting  and  budget 
documents  Also,  wc  conducted  interviews  with  program  managers,  analysts,  and 
technicians  responsible  for  the  acquisition  and  deployment  process  of  the 
THINTHREAD  and  TRAiLBLAZER  programs  Specifically,  we  reviewed  the 
extent  to  which  the  NSA  considered  the  use  and  deployment  of  the 
THIN  THREAD  system  to  satisfy  an  urgent  security  requirements  need. 

(U)  We  performed  this  audii  from  April  2003  through  May  2004  in  accordance 
with  generally  accepted  government  auditing  standards 

•  • 

(Li7FO(JO|  Limitation  to  Scope.  Because  ouc  objective  was  limited  to 
evaluating  a  Defense  Hotline  allegation  relating  to  the  acquisition  of  the 
TRAILBLAZER  and  TH INTHREAD  programs,  we  did  not  review  the 
management  control  program  .| 


fU)  Uae  of  Computer-Processed  Data.  We  did  not  use  computer-processed 
data  to  perform  this  audit. 

Use  of  Technical  Assistance.  Audit  Follow-up  and  Technical 
Support  Personnel  assisted  us  during  the  audit  The  Technical  Assessment 
Division  assisted  in  the  analysis  of  NSA  acquisition  and  deployment  of  Signals 
Intelligence  systems. 

(U)  General  Accounting  Office  High-Risk  Area.  The  General  Accounting 
Office  has  identified  several  high-nsk  areas  in  DoD.  This  report  provides 
coverage  of  the  Infrastructure  Inefficiencies  and  Information  Technology 
Investments  high-nsk  areas 

(U)  Prior  Coverage 

(U//FDfcfQiPuring  the  last  5  years,  the  Inspector  General  of  the  NSA  issued  one 
report  oo  TRAILBLAZER  that  discussed  improperly  based  contract  cost 
increases,  non-conformance  in  the  management  of  the  Statement  of  Work,  and 
excessive  labor  rates  for  contractor  personnel  No  prior  coverage  has  been 
conducted  on  the  NSA  TH INTHREAD  Program  during  the  last  S  years. 
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J  (t//tJOW€U.Report  No.  ST-03-0014.  ‘TRAIL-BLAZER  l/SIGINT  Programs 

Systems  Engineering  and  Technical  Assistance  Contract."  September  30.  2004 
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Appendix  B.  TRA1LBLAZER  (U) 


(U)  Objectives  of  TRAILBLAZER  and  the  Technical 
Demonstration  Platform 


(li/li0UO>  The  following  is  a  list  of  new  capabilities  to  be  provided  by  the 
TRAILBLAZER  Technical  Demonstration  Platform.  ; 


(U'/P#y®)  The  use  of  a  scalable,  integrated,  plug-and-play  framework  will  allow 
the  Signals  Intelligence  System  to  operate  with  foe  speed  and  flexibility  required 
to  follow  NSA  targets  in  today’s  modem,  global  network  environment. 

(gflSIl  Specifically,  the  TRAILBLAZER  framework  must  leverage  the  best 
commercial  technology  and  practices.  It  also  must  be  standards  based  to 
accommodate  mission  applications  as  defined  by  the  mission  managers  and 
scalable  to  meet  future  challenges  that  were  pot  well-defined.  I  | 
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(U)  Operational 
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(V)  Processing  Elements.  Five  processing  dements  arc  needed  in  the 
architecture: 
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Appendix  F.  Technical  Studies  (U) 


(U/AF0TO) 
Technical  Review 


/  THINTHREAD 


(UA'RMJO)  The  purpose  of  the  ^  •.  |/THlNTHRhAi> 

Technical  Review,”  Febniarv  ?001  and  the  'l  ■  ,  1/ 

THINTHREAD  Follow-up  for  D/DIR,"  March  2001,  was  to  identify  similarities 
and  differences  between  TRA1LBLAZER|  |i  and 

THINTHREAD.  Areas  assessed  were  performance,  resources  maintainability, 
scalability,  and  the  ability  to  meet  mission  requirements.  The  study  showed  that 
there  were  significant  overlaps  in  effort  because  both  programs  were  solving  the 
same  mission.  The  major  boundaries  were  similar,  but  the  approach  and 
philosophy  differed.  The  following  was  extracted  from  those  reports. 
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I  N&A  1mm  not  executed  Ac  February 

2001  guidance  by  the  Deputy  Director  to  provide  development  and  deployment 
resources.  For  example,  the  TH INTHREAP  innovation  laboratory  was  out  of 
funds  even  though  it  was  promised  I  lo  continue  development.  Also, 

TH  INTHREAD  contractors  were  let  go  and  ito  further  progress  had  been  made. 
There  was  a  shut  down  of  TRAILBLAZER  learning-opportunities  and  there  was  a 
termination  of  alternative  TH  INTHREAD  approaches  to  the  mission. 
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(U/^OVOy  Summary  of ! 

list  of  recommendations  of  the| 
Technical  Review. 


•  (U//F0W01  Provide  requested  resources  to  the  THINTHREAD  team 
(dollars  and  people) 

•  (TJ//P0PPV  Pro  vide  THINTHREAD  proper  management  support 

•  (U//WW0^  In  light  of  this,  not  all  of  thJ 

system  is  required  L  " 

THINTHREAD  has  the  capability  to  petfonn  the  digital  network 
exploitation  mission  The  digital  network  exploitation  operational  mission  could 
be  completed  entirely  with  |  _  I 
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(U)  NSA  Modernization  Follow-Up  Study 
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*  '  TSrrSMhc  “NSA  Modernization  Study/’  July  13,  2001,  and  the  “NSA 

Modernization  Follow  Up  study/*  December  2001,  stated  that  THfNTHRfcAD, 
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y&HU4  The  stu4y  stated  that  the  must  interesting,  development  in  the 
THINTHREADstory  that  occurred  between  the  two  studies  was  the  i 


given  additional  resources  and  access  to  the  analytic  community,  the  “who.  what, 
why”  equation  may  have  been  refined  earlier 
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H  The  main  advantage  of 
developing,  testing  and  deploying  it;  it 
mnuirina  only  hardware  ami  mi 


tii  gitj'neWcIrk  mi« U igmce  development  work  and  benefit  I 
\  Ail  would  be  provided. 

(U//PttUQ)[ _ 

report  stated  that  although 
optipn,  it  is  not 


appear  to  be  an  attractive 


(U//rOUO)|  |ia  not  flexible.  In  addition,  it  is  a  complete 

system  that  is  inflexible  and  forces  users  to  interact,  task  and  use  it  in  a  certain 
way.  This  means  that  would  be  extremely  difficult  foij  *  **"] 
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3  (IT)  A  complete  1  yumea  that  solve*  ell  die  processing  problems  tfadf  ll  bee  it*  amm  input,  control,  statu*, 
monitoring,  nary  interfaces  and  data  fonnatt  Even  though  other  system*  have  exactly  the  seme  problems 
to  solve,  stove-piped  systems  wtl]  implement  their  own  solutions  to  each 
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(U)  Preliminary  deployment.  If  required,  the  followuig  components  are 
recommended  for  a  preliminary  deployment: 
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Appendix  G.  Deployment  Studies  (U) 


(LI)  THINTHREAD:  A  Deployment  Suitability  Assessment 


purpose  of  the  Deployment  Suitability  Report  and  the  assessment  it 


READ 


(U)  Preliminary  research  on  THINTHREAD  wan  performed  by  the  Test, 
Technology  and  Evaluation  team  in  February  2002.  but  the  formal  evaluation  was 
executed  from  March  1.  2002.  through  March  18.2002. 

(U)  Summary  of  THJNTHREAD  Deployment  Suitability 

(UfttaUQ)  TH INTHREAD  was  successfully  deployed  for  developmental 
evaluation  to  a  single  pilot  site  with  approximately,  a  dozen  analysts  using  the 
prototype.  Since  then,  the  Signals  Intelligence  Directorate  identified 
THJNTHREAD  as  an  operational  prototype  that  merits  further  assessment  as  an 
analytic  tool  that  may  improve  analysts’  ability  to  deliver  quality  Signals 
Intelligence  to  their  respective  customers  In  preparing  for  the  next  step  of 
deploying  THINTHREAD  to  a  diversity  otl  Isitca.  the  TH INTHREAD 
Program  team  has  made  progress  in  identifying  and  executing  the  planning  and 
coordinating  activities  that  arc  necessary  for  future  deployments.  However,  those 
activities  had  not  been  executed  to  the  level  needed  for  future,  efficient  and 
successful  multiple  deployments  Prior  to  deploying  THINTHREAD  any  further, 
die  Test,  Technology  and  Evaluation  Team  advised  that  the  following  risk 
mitigation  and  preparation  activities  be  addressed 
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(l  )  General  Recommendations. 

(U/<J*0nO)  The  Deployment  Suitability  Assessment  Study  made  the  following 
recommendations: 
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(U)  minimize  operational  risk  of  deploying  TH INTHREAD  to  multiple 
I  jtites  by  providing  training  to  site  support  personnel,  implementing  a 
TH  INTHREAD  systemwide  configuration  management  process,  and 
developing  a  process  by  which  site  personnel  can  identify  and  resolve 
problems  with  the  system  as  they  arise; 

fU)  generate  user,  system,  technical,  and  programmatic  documentation. 
Existing  documentation  is  insufficient  to  sustain  the  prototype  for  the 
planned  length  of  deployments; 

(U)  accelerate  deployment  planning;  it  has  bean  slow  to  occur  Develop 
necessary  partnerships  with  recipient  |  frites  and  deployment  support 
organizations;  and 


•  (U/)T0UO|  acquire  sufficient  numbers  of  appropriately  skilled  personnel 

to  develop  documentation,  perform  deployment  planning,  support 
deployments  to  |  kites,  automate  user  support  processes,  and  develop 
controlling  and  measuring  activities  for  multiple  deployments  of 
TH  INTHREAD 


(U)  For  further  details,  listed  below  are  the  seven  key  findings  and  related 
discussions  in  die  THINTHREAD  Deployment  Suitability  Report 


(U)  Finding  1  Discussion. 

(U/TOUQJ  There  were  no  technical  issues  that  would  prevent  THINTHREAD 
from  being  deployed  as  an  operational  prototype. 


•  (U)  Current  THINTHREAD  system  development  procedures  were 

effective,  and  a  cooperative  development  effort  was  proceeding  smoothly 
with  the  pilot  site. 
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fU)  Recommendations.  There  were  no  recommendations. 

(U)  Finding  2  Discussion 

(U/#«UttfrTlie  overall  operational  risk  of  making  successful  deployments  was 
moderate,  when  viewed  from  an  operational  prototype  perspective  The  Finding  2 
discussion  areas  were  divided  into  two  parts:  I )  those  items  that  would  increase 
the  operational  risk  of  deploying  TH INTHREAD;  and  2)  those  items  that  would 
not  increase  the  operation  risk  of  deploying  TH  INTHREAD 

(U//PM1Q)  Part  1.  The  following  would  increase  the  operational  risks 
associated  with  deploying  THINTHREAD: 


•  (U)  lack  of  site  support  documentation  and  training  for  site  support 
.•  personnel. 

•  (U)  lack  of  a  configuration  management  process  for  THINTHREAD. 

p  (U)  absence  of  a  method  for  site  personnel  to  identify  or  resolve  a  problem 
with  the  system. 

(U)  Recommendations.  See  the  recommendations  for  THINTHREAD  Findings 
3  through  ?  to  address  these  risks. 

(U//FOUOJ  Nrt  2.  Few  analysts  were  depending  on  THINTHREAD. 

Additional  deployments  would  not  interfere  with  the  benefits  currently  being 
enjoyed  by  the  few  analyst  users  However,  by  adding  users,  certain  issues  would 
become  more  prominent  Those  issues  were: 


•  (U)  lack  of  analyst  training  on  Web-interface  and  equation  development 
for  THINTHREAD; 

•  (U)  no  method  for  THINTHREAD  personnel  to  assess  user  feedback  on 
the  Web  interface; 


•  (U)  no  way  to  better  measure  the  value  of  THINTHREAD  to  analysts; 


•  (U)  some  THINTHREAD  files  could  not  be  viewed  to  pursue  corrective 

actions. 
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(UaTOUQ)  Rtcominnditiooi.  The  issues  previously  mentioned  would  not 
prevent  the  deployment  of  THINTHREAD.  however,  it  was  recommended  that 
investigation  in  to  those  areas  be  performed  in  order  to  improve  the  operational 
value  of  THINTHREAD. 

(U)  Finding  3  Discussion 

(LI)  There  was  minimal  user,  system,  technical,  or  programmatic  documentation 
The  existing  documentation  was  insufficient  to  support  the  planned  deployments. 

•  (U)  Their  was  no  user  documentation  on  how  to  use  the  tool  That  was 
inefficient  and  could  lead  to  a  significant  increase  in  user  error, 
Difficulties  using  the  tool  can  also  lead  to  an  increase  in  support  calls  that 
drained  project  resources. 

•  (U)  THINTHREAD  did  not  have  s  clearly  staled,  documented  set  of 
mission  goals  or  objectives  There  was  no  programplan.  making  it 
difficult  to  comm  uni  calc  information  about  THINTHREAD  at  a 
programmatic  level 

•  (U)  A  detailed  schedule  including  all  activities  related  to  the  system  and 
deliverables  before,  during,  and  after  deployment  was  not  available.  A 
high-level  schedule  containing  the  chronology  of  deployment  was 
developed,  but  a  detailed  schedule  was  essential  for  proper  resource 
management,  costing,  and  prediction. 

•  (U)  No  installation  and  maintenance  documentation  was  completed  for 
site  support  personnel,  although  the  team  was  working  with  the  pilot  site 
to  develop  installation  documentation.  Lack  of  installation  and 
maintenance  documentation  endangered  the  continuity  of  operations  at  the 
deployment  sites 

I 

•  (U)  There  was  no  operational  support  documentation  for  site  personnel 
Lack  of  support  documentation  could  adversely  affect  timely  correction  of 
problems  and  oouid  lead  to  loss  of  collection,  loss  of  processing,  or  loss  of 
data  and  metadata 

(U)  The  effects  of  minimal  documentation  could  be  overcome  if  an  integrated 
team  accomplished  the  development  and  deployment  of  THINTHREAD. 
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(U'/BttySy  However.  the  risk  represented  by  insufficient  doc 
greatly  increased  because  TH  INTHREAD  was  expected  lo  remain 
multiple  sites  for  1  to  2  yean. 


(U//FOUCU  Recommendations.  Develop  necessary  documentation.  That  would 
necessitate  an  increase  in  the  personnel  devoted  lo  documenting  the  system,  but  is 
essential  to  the  successful  deployment  and  operation  of  THlNTTIREAD  The  lack 
of  this  documentation  presented  a  serious  hale  lo  the  successful  deployment  of 
TH  INTHREAD 

(U)  Finding  4  Discussion 

(U//FOUO  >  Deployment  planning  was  slow  to  occur,  and  necessary  partnerships 
with  recipient  |  bites  and  deployment  support  organizations  were  not 
developed 


■J 

"J 

•  (I  f 'TTHIQjJ  liiionrilljf.  the  TH  INTHREAD  Program  Team  focuaed  on 
the-prototyne  demonstration  at  the  pilot  site.  The  program’s  scape  did  not 
include  multiple  deployment  sites  and  support  activities  Because  the 

TH  INTHREAD  team  received  requirements  from  the  Signals  Intelligence 
Directorate  for  TH  INTHREAD  deployment  planning  and  to  multiple  and 

1  locations,  the  TH  INTHREAD  Program’s  scone 
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began 


the  numbers  and  types  of  skills  needed  to  plan  and  carry  out 
and  support  activities  by  producing  a  Staffing  Support  Plan. 

•  •  *• 

(lWFW©)Tbe  TH  INTHREAD  Program  Team  did  not  have  a 
comprehensive  Planning  process.  The  lack  of  planning  activities  with 
"Jtitef  and  deployment  support  organizations,  including 
1  wwmmanun  ana  planning  for  partnership*  and  agreements,  may  have 
delayed  deployment  efforts  and  increased  deployment  costs 


was  an  interim  capability  and  was  scopea  as  a 
and  scalable  system. 


T  TI UNTHREAD 

stamulr 


Hone,  self-contained. 


(U//POUO)  Work  roles  essential  to  system  deployment  and  operation 
were  not  identified  for  multiple  site  deployment  scenarios.  The 
relationships  between  work  roles  throughout  the  system’s  life  cycle  were 
not  defined.  The  authorities  for  each  work  role  had  not  been  identified 
The  organizational  structure  set  forth  by  the  TH  INTHREAD  Program 
Team  focused  on  system  development,  and  did  not  address  system 
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(U//POWO)  deployment  or  operation  in  al  I 

environment.  The  THINTHREAD  Program  Team  was  defining  the 
additional  work  roles  necessary  for  deployment  activities 

•  (U/4MUQ)  No  formally  recognized  prioritization  procedures  existed  to 
address  the  variety  of  requirements  that  would  come  to  the  program  offiot 
from  the  numerous  pites  when  TH  INTHREAD  is  deployed. 

•  (II)  THINTHREAD  did  not  identify  all  the  organizations  requiring 
coordination  through  the  various  stages  of  deployment. 


(II//totiQ^RetofimeiidatkMif.  The  findings  presented  a  serious  risk  to 
successful  deployment  of  THINTHREAD  To  mitigate  those  risks,  the  following 
recommendations  were  suggested. 
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■9  (U)  Develop  deployment  plans  in  partnership  with  specified  recipient 
sites 

•  (U)  Coordinate  and  plan  deployment  activities  with  deployment  support 
organizations. 


•  (LI)  Develop  and  establish  organizational  processes  that  could  support 
multiple  deployment  activities,  such  as  a  requirements  management 
process  to  assist  in  prioritizing  program  activities. 


•  (U)  Continue  addressing  the  work  roles  needed  to  plan  and  carry  out 

deployment  and  support  activities  by  producing  a  Staffing  Support  Plan. 


(U)  Finding  5  Discussion 


(UMFQUO)The  customer  service  and  user  support  processes  for  THINTHREAD 
were  manually  intensive  and  directly  involved  the  technical  staff. 

*•  *,  . 

•  (D/TOUO)  The  quality  of  service  and  support  received  by  analysts  using 
THINTHREAD  was  inconsistent.  Some  of  the  analysts  interviewed  by 
Test,  Technology  and  Evaluation  Team  viewed  the  THINTHREAD 
Program  Team  to  be  unresponsive  in  general  customer  support,  while 
other  analysts  experienced  good  .responsiveness  to  (heir  requests  for 
additions  or  modifications!  "I 


•  (U//FOUO)  There  was  no  formal  trouble  ticket  mechanism  in  place.  If  a 
user  or  operator  had  a  problem  with  THINTHREAD,  be  contacted 
someone  on  the  THINTHREAD  Program  Team  This  contact  depends  on 
personal  acquaintance,  because  no  list  of  THINTHREAD  personnel  was 
readily  available,  and  there  was  no  mechanism  to  locate  the  proper 
THINTHREAD  contact  Similarly,  the  status  of  the  response  to  a  request 
could  only  be  learned  by  personal  contact. 

•  (U/TOW)  There  was  no  formal  mechanism  for  managing  externally 
generated  requirements,  to  include  submitting  a  new  requirement  to 
THINTHREAD  or  for  following  the  progress  of  the  work 
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•  (U/ifiJUOH  single  person  on  the  THINTHREAD  team  was  in  charge  of 
maintaining  the  |  1  •"  •-  1  With  just  a  few  users, 

(hat  person  was  Biff  u>  aner  nr  itinmre  I  hecordinv  to  user 

requests- received  On  the  telephone  orVy  e-mail.  However,  as  more  users 
Be  added,  one  person  may  have  difficulty  meeting  the  increased  demand 
The  lack  of  personnel  .may  become  a  problem  as  THfNTHREAD  was 
Scaled  to  handle  an  increased  user  load  and  I  l 


•  (U//POUQ)  A  list  of  Web  interface  functionalities  proposed  by  analysts 
was  not  incorporated  into  THfNTHREAD 


(IWODQl  Recommends  lions.  The  findings  that  led  to  the  recommendations 
would  not,  have  prevented  the  successful  deployment  of THfNTHREAD  to 
I  I  The  recommendations  were  made  to  improve  the 

operational  efficiency  of  THfNTHREAD  after  it  was  deployed  atf" 
sites.  1 


•  (U)  Institute  an  automated,  formal  THfNTHREAD  customer  service  and 
user  support  process 


•  (U/TOUOt  Acquire  and  use  a  trouble  ticketing  tool,  which  can  manage 
externally  generated  requirements  as  well  as  system  problems.  This  tool 
should  interact  with  the  SID  requirements  process  The  THfNTHREAD 
Program  Team  should  address  high-priority  requirements  that  fit  within 
the  program  scope 

•  fll)  Develop  a  process  to  address  Web  interface  functional  enhancements 
requested  by  analysts. 

i  *  * 

•  (U//rOt*H  Address  the  lack  of  personnel  depth  in  the  THfNTHREAD 

Program  Team  for  maintaining  l  las  THINTHREAD  is  scaled  to 

handle  more  users  and  multipie  site  deployments. 


(U)  Finding  6  Discussion 


(L/TPOtJO)  There  was  no  process  for  controlling  and  measuring  operational, 
programmatic,  and  technical  performance  effectiveness  of  THfNTHREAD.  once 
deployed 

•  (U//F©U^)  THINTHREAD  did  not  have  a  clearly  staled  set  of  system 
specifications  or  a  method  to  validate  the  THINTHREAD  concept  or 
performance  results  Thus,  there  was  no  way  to  determine  if  the  system 
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(W  'FW9t  was  successful  or  if  it  failed,  which  may  lead  to  confusion  in 
stakeholders’  understanding  of  the  success  of  THINTHREAD. 

•  (U/TOUONTH  INTHREAD  had  mechanisms  for  assessing  progress  and 
status  within  the  system  development  phase  However,  there  were  no 
systematic,  consistent  mechanisms  for  overall  program  measurement 
which  included  identifying  test  criteria,  establishing  program  schedules, 
and  documenting  a  system  baseline  In  address  progress.  No  feedback 
mechanisms  had  been  identified  such  as  risk  management  or  contingency 
planning 

■  (U//POU0)  Aside  from  schedule  control,  there  were  no  tools  to  measure 
progress  towards  program  objectives  because  there  were  no  formalized 
program  objectives. 

•  (TJ/TOU0)  TH INTHREAD  did  not  have  Standards  of  Performance 
established  for  personnel  developing  THINTHREAD.  If  future 
THINTHREAD  development  crosses  organizational  groups,  a  common  set 
of  Standards  of  Performance  for  personnel  may  beoome  necessary 

•  (L7/TQUQ)  There  was  no  configuration  management  process,  which  could 
pose  a  risk  when  reporting  system  problems  and  correcting  them  at  a  site. 

In  addition,  the  lack  of  a  configuration  management  process  could  cause 
difficulty  when  upgrading  or  deploying  ventona,  and  alio  make  rollback 
extremely  risky 


•  (U/7POU©)  The  fault  or  warning  detection  w«  siitomated  hm  the _ 

necessary  reactions  and  responses  were  not  P 
software  provided  a  simple  way  to  check  status  of  various  system 
parameters  through  a  Web  interface  However,  when  an  operator 
recognizes  a  problem,  be  must  acbveJyihidate  corrective  measures. 
Sometimes  that  action  is  a  simple  matter,  such  aa  restarting  a  process.  At 
other  tunes,  physical  hardware  must  be  changed  No  corrective  action 
could  be  cxecutod  rf  an  operator  was  not  present  to  initiate  a  corrective 
action. 


Y3ATO50  U5C  3605 


•  (U//POW0I  THINTHREAD  had  no  established  process  for  managing 
requirements. 


(U/*6tK*Recoi 

would  ntit  have 


dadoes.  The  findings  that  led  to  the  recommendations 
the  successful  deployment  of  THINTHREAD  to 
The  recommendations  were  made  to  improve  the 
of  THINTHREAD  after  it  was  deployed  j 


tes.  The  following  is  a  list  of  recommendations. 


•  (U//FO&GQ  Develop  processes  and  acquire  automated  tools  based  on 
clearly  stated  specifications  for  controlling  and  measuring  die 
programmatic,,  operational,  and  technical  poformance  of  THINTHREAD 
deployment  [  Hsites 
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•  (U//TOUO)  Acquire  a  commercially  available  configuration  control 
system  to  enable  versioning,  rollback,  and  configuration  management 
The  system  should  interface  with  the  trouble  ticketing  system  and  the 
requirements  management  system  to  provide  full  traceability 

•  (U/flFOlOTl  Develop  Standards  of  Performance  (if  future  THINTHREAD 
development  crosses  organizational  groups). 

•  (U/-TOUO)  Automate  reactions  and  responses  to  the  existing  fault  and 
warning  detection  system  to  improve  successful  deployment 


•  (UTTOUQ)  Establish  a  requirements  management  process  for 
THINTHREAD  that  is  linked  to  the  SID  requirements  management 
pniccss. 


0 

rJ 

rJ 

rj 

u 

G 

j 


(U)  Finding  7  Discussion 

(U//EWQ)Tbe  number  and  skill  mu  of  TH (NTH READ  personnel  was 
inadequate  to  support  the  proposed  number  of  deployments;  however,  personnel 
resources  for  supporting  future  multiple  deployments  were  being  identified. 

There  were  shortfalls  with  the  numbers  ana  skills  of  personnel  requited  for 
planning  and  executing  installations  at  candidate  sites.  Also,  there  was  a  lack  of 
documentation,  providing  the  necessary  maintenance  and  hdp  desk  support, 
configuration  management,  and  planning  and  scheduling  Collectively,  those 
shortcomings  created  a  serious  nsk  to  success  fill  completion  of  multiple 
deployments. 

_  _  _  « 

(1//POUO)  Recommendations.  The  finding  presented  serious  risk  to  successful 

deployment  of  THINTHREAD.  It  was  recommended  to  continue  to  identify  and 
acquire  the  number  and  types  of  skilled  personnel  resources  required  for 
documentation,  deployment  planning,  actual  deployment  tJ  Men,  and  an 
automation  of  user  support  processes  and  controls  that  measures  THINTHREAD 
activities  to  reduce  that  risk. 


J 

J 

J 

j 

J 


It 

Tor  seenETfleeMi  mtmtn  tt* 


Bates  000296 


1>IH  IM  ti^K***** 


USA  M3)  50  U5C  3605 


TOt»  HilVRWrMi'OMimm****  1M 


N§A(bmH-4(d  (bK3)5flU5C 

J605 


Appendix  H. 


^ _ |and 

THINTHREAD  -  a  Complementary 
Approach  (U//FOU0) 


(U)  This  appendix  presents  excerpts  from  the|  land 

THINTHREAD  white  paper 


(U)  Introduction 
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tGtfUlThc  purpose  pt  the  | _ 1  and  THINTHREAD  white 
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(U)  System  Capabilities  Overviews 
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{U//KQUCM  The  following  section  outlines  the  strengths  of  each  system  in  its 
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'  (U)  T1  UNTHREAD  Bnefin*.  23  May  2002.  Microsoft  PowerPoint  Slide  28 
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(U)  Optimal  System 
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(U)  Cost  Analysis 

(17)  The  following  hypothetical  scenarios  illustrated  how  a  combination  of  both 
system)  may  produce  a  more  effective  result  in  terms  ofboth  complexity  and  cost. 
A  few  assumptions  apply: 
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(U)  Conclusion 


(ewe*  The 


, - r  TH INTHREAD  systems  each  pots— ed 

unique  strengths  that  could  be  applied.to  digital  network  exploitation 
transformation.  I 


That  approach  had  the  potential  to  produce  a 

dent  collection  system. 
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Appendix  I.  Glossary  (U) 


(U) 


Coiuultativr  Committee  for  litenutioul  Telegraph  aad  Telephone  (CCITT). 
Consultative  Committee  for  International  Telegraph  and  Telephone  is  the  International 
organization  responsible  for  the  development  of  communications  standards  Now  called 
the  International  Telecommunication  Union  Telecommunication  Standardization  Sector. 
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laltrutloDtl  Telecommunication  KiIod  Telecommunication  Standardization 
Sector  (ITLJ-T).  International  body  that  develops  worldwide  standards  for 
telecommunications  technologies.  The  International  Telecommunication  Union 
Telecommunication  Standardization  Sector  carries  out  the  functions  of  the  former 
Consultative  Committee  for  International  Telegraph  ind  Telephone 


# 

International  Organization  for  Standardization. (ISO).  International  organization  that 
is  responsible  for  a  wide  range  of  standards,  including  those  relevant  to  networking.  The 
International  Organization  for  Standardization  developed  the  Open  System 
Interconnection  reference  model,  a  popular  networking  reference  model 
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Institute  of  Electrical  and  Electronic*  Engineer*  (IEEE|M2_L  Institute  of  ElectncaJ 
and  Electronics  Engineers  LAN  protocol  that  specifies  an  implementation  of  the  physical 
layer  and  the  MAC  sublayer  of  the  data  link  layer.  Institute  of  Electrical  and  Electronics 
Engineers  B02.3  uses  Carrier  Sense  Multiple  Access  Collision  Detect  access  at  a  variety 
of  speeds  over  a  variety  of  physical  media.  Extensions  to  the  I  Institute  of  Electrical  and 
Electronics  Engineers  802.3  standard  specify  implementations  for  Fast  Ethernet 


Open  Systran  Interconnection  (OSI).  International  standardization  program  created  by 
International  Organization  for  Standardization  and  International  Telecommunication 
Union  Telecommunication  Standardization  Sector  to  develop  standards  for  data 
networking  that  facilitate  multi-vendor  equipment  interoperability 
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Appendix  J.  Response  to  Managementp 
Comments  (U) 


<0f/9f)  Management  Comments. 


:  Office  of 


i.  A  significant  point  of  departure  among  O 
Inspector  Genera)  Report  assertions  and  NS  A  understanding,  reflected  within  NSA’s 
responses  below,  concerns  the  validity  of  using  THINTHREAD  as  the  Agency's  Digital 
Network  Exploitation  system.  NSA  understands  that  THINTHREAD  does  not  contain  the 
functionality  or  capability  maturity  to  be  the  InteJIn/ence  Community's  digital  network 
exploitation  system  and  it  does  not  need  to  be.  I  1  is  the 

Community's  digital  network  exploitation  system-,  it  contains  the  depth  and  breadth  of 
capabilities  to  exploit  all  known  digital  network  exploitation  targets  (sec  Annex  A),  it 
uses  ott-hnnlnpigt  (both  in  hardware  and  software),  and  it  supports  itself 

with  I  jprogram  management  and  systems  engineering.  THINTHREAD’s 

best -id-breed  capabilities  are  integrated  into  |  Valuing 

THINTHREAP's  contribution  to  digital  network  exploitation  ana  capitalizing  on 
|  | s  programmatic  and  systems  engineering  maturity.  Any  assertion 

mat  i  hin  i  MKhAU  might  be  used  to  “replace”  ]  /  [  exposes  a 

significant  misunderstanding  of  the  situation.  1  1 


Audit  Response.  Two  recent  Joj 
mission  support  equipment  stated  that,  _ 
technical  capability  to  |  ' 

mission  requirements.  i  rilNTHRhAD  hasbeftj  able  to  success 


aJuations  of  NSA  sites  and 
does  not  have  the 
ho  fully  meet  NSA 
xsTullyl 


MI,,,,,  .  ;  I 

since  November 2000.  Also,  THIN  I  HKfcAD  can  be 
uirements  THINTHREAD  originally  concentrated  on 

]aDd£auldJuve  been 

1  provides  for  a 


VSA  (b){1 )  1  4(c),  (b)(3)  50  use  3605 


stateo  tnaf  i  hi.'n  i  ttKt-.ALi  developers 
no  further  progress  was  made  toward  developing  THINTHREAD.  As  a 
result,  TRAILBLAZER  learning  opportunities  were  shut  down  and  alternative 

THINTHREAD  approaches  to  the  digital  network  exploitation  mission  were  terminated 

#•»  • 

•*••  •  * 

~i€i  Management  Comments.  Although  NSA  will  agree  to  conduct  an  independent 
asscssmimt  Of  TRAILBEAZER  and  tHlNTHREAD;  we  believe  that  it  will  validate  all  ol 
the  assertions  made  in  our  responses  to  the  Office-of  Inspector  General  recommendations 
Furthermore,  vye  do  not  agree  with  the  original  Hot  Line  complaint  that  fraud,  waste,  and 
abuse  occurred  during  the  T  HINTHREAD^  • '  [TRAIL  BLAZER 

decision  process,  which  wgsconducted  in  mi  open  jasmon  to  provide  the  best  benefit  for 
Agency  mission  and  customers  More  importantly,  there  has  been  no  evidence 
documented  during  (his  Office  of  Inspector  General  investigation  that  supports  dial 
complaint.  **;  /„*’/ 

Qudli  Response.  'Die  report  documents  that  NSA  spent  approximately!  I 
>  develop  tby  TRAlLBLA^ER  Initial  Transformation  Activities  even  chough 
nd  e?tem^  reports  recommended  using  THINTHREAD  *nd  THINTHEAD 
lechnologms-as  thdinlerirodolulion  for  digital  network  exploitation  fhc  report  also 
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(St  Management  Comments.  The  world's  most  advanced  erypuJldgil  experts,  whe  Jtic 
resident  at  NSA.  have  assessed  THINTHREAD  and|  “<■" 1  ^nd  arc  ; 

confident  in  the  approach  taken  With  mission  UU^gJlJMilg  Uimj  tie  to  NSA. 

these  experts  arc  best  suited  to  moke  on  assessment  of  the  patipti  'A/cryptn  logic  needs  ;  *. 

V  .  «V  ••  ,  • 

•  ♦  «  ♦ 

iUHHudil  Response.  The  report  accurately  documents  out  aisessrruhit  of  all  of  thb 
information  received  during  this  audit  including  intervicsys  aiid' information  received; 
from  NSA  cryptologic  experts  and  current  and  former  NSA  employees.  Who  are  expOts 
on  THINTHREAD  and)  iBascd  oh aaiflim  and  evaluations  that  • 

NSA  performed,  we  believe  that  NSA  management  isatwarc-Urti  jj  :  I 

does  not  meet  the  mission  requirement  |  ;?•  I0MT 

that  THrNTHREAD  could  be  more  cosl-cHeqtive  sotution  that?  splutioris1, currently  ofing 
pursued  However.  NSA  management  has  refused  tfi  consjidn*  that  THINTH  ft  F. AD  i>  a  . 
viable  solution  to  the  Intelligence  C  (immunity  s  digital'  network  exploitation 
requirements  It  is  our  opinion  that  THlhfTHREAD  obuld  he  more  cost-effective  sohitiop 
than  solutions  currently  being  pursued  Aji  external  an»l  inrimenH«-ni  studty  should  b< 
conducted  to  compare  THINTHREAP  and  | ! |  to  validate  the  ;; 
capabilities  of  both  systems.  '•  I  ’• 


(^Management  Comments 

NSA 

Coi 


wot 


the  current  digital  network  exploitation  architecture 


FH INTHREAD  capabilities  art  already  deployed  wjthw  : 

as  a  result  of  previous 
„ .  . pommop 'sense  jl 

i non  THINTHREAD  capabilities  from  ; 


•  .  *  *  it  i  l  |  #  ♦  *  J  ** 

(SABIt  Audit  Response.  FH1N  I  HREAD  c^i^li^  alTjioi  being  hilly  used  within  the 


existing  NSAf 
system  that  is  capable  ol 
complementary  approach  usir 
THINTHREAD  is  enhanced  t 

(MR  Management  Coipm 

Technical  Review."  written 
THINTHREADst 


_ i _ ^ _ ^  uted _ 

THINTHREAD  is  the  oqly  developed 


~1 

and  THI 
ommau 

"  IMPdwWvaWTTS 

■  ■  And  7  HIN'THR RAD'S 
INSa  agreed  and  leVpragdd  AnAlLBt.AZER  Activity 


PH  READ 

1  adopting 


] 


land  THnVtHKf  AP  Technical 
maintainability,  ycalability. 
and  the  ability  to  meet  mission  requirements.  Tfrd- study- roccjmmondcd  adopting  hiid 


(3*91)  Audit  Response.  The  j  •'  •'  „ _ ,  _ , . 

Review.''  only  reviewed  the  areas  ot  perform  ah  ct,  resources,  maintainability.  >cal 


i  HISm  lRt  AD  In  addition,  the  review  slated  that 
operational  mission  could  be  completed  entirely  with 


me  wqpnoq'WK  ny  fwawyifl  a^iav 

t  Digital  NetWork  Exploitation 
ith  THIN  ITlftLAD  if  (hclUr; 


|aa| 

£U 


 that  THINTHREAft  could  not  be  integrated  mto  the  I  :  I 

”1  system  because  the  fundamental  approach  and  iachpical  design  were  not 
'»mpmnie  .The  “NSA  Modcmi  ration  Follow-Up  Study"  stated  that  (he  NSA  attempt  to 
merge  rHfN  lTfREAD  technology  into  the  Architecture  based  00  tbfl  I 

I  ~~~lservice  was  not  successful  and  was  not  technically  faakrple  oecause  when 
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TSWfl  THfNTHREAD 
currently 


quickly  and  cheaply  would  be  possible 

^  ^  I  Management  Comments-  In  FY  2002. 
THINTHREAD  to  I  biles  NSA  complied  with 
THINTHREAD  Deployments!!  * 


1&MQfr  Audit  Response.  NSA  delayed  the  dentoyrntpiCofTHlNTHREAp  The  FY 
2001  Intelligence  Authorization  Bill  (CMIS  1 :2S-2001);  December  1 7.  2001 ,  by  the 
House  •Permanent  Select  Committee  qrr  Intelligence  staled  the  conferee*  directed  that 
-  " - rTftfimn - 


contracted  immediately  for 


IKEAP  ^BDlDWDBpU  it  if  wtari 


the  conferees  strongly  recomlrnetOi 


i'CTiSuproiEngitfiii»^mi;*  ^turmaiiL 


Sf  deploy _ 

1  to  pnly  I  I 
arch  ^OyTTUie 


(JiAttibNSA  briefed  the  House  Permanent  Select  Committee  on  Intelligence  that  it  would 

but  thp  THIN THR.EAD  Project  Management  Office 
s  congressional  direction  to  immediately  deploy 
NSA  had  depfpycd  THJNTHfcEAD  | 

sites,  withi  kite  deployments  occurring  sincttNovcmbcr  2 QOS;  In  March ; 
THINTHREAD  deployment  process, tMttfgTfid  W  lhfilT_ 

I  knanaucmeni  office  and  thef  ••  iPmgr^n  Mariitger  al^o 

became  the  Program  Manager  responsible  for  the  operational  deployment  ojf 
THINTHREAD  Although  NSA  nad  a  contract  available  to  immediately  deploy 
THINTHREAD  as  directed  by  Cdnuress.  NSA  spread  die  deployment  over  a  2  ycar- 
penod.  We  also  determined  that!  I  tasked  ni  THISTHkiAii  vyerc 

Siisncci  rm.M  HRI  All  wax  not  ilmtovetil  •  •'  ||a|>  ^  thc 

iHlNlHKhAD  was  only  depioyedln^"^™^^"^^  «  T*  — | 

sites  1 

f  a  1  • 

•  f 

NSA  did  hot  plan  to  adequately  lest  or  maximize  the  performance  of; 
THINTHREAD  and  did  not  encourage  analysts.to  use  THINTHREAD.  In  addition.  NSA 
provided  inaccurate  information  to  the  audit  team  when  it  suited  that  no  analysts  or 
personnel  were  at  mlloriinn  sire*  in  nrnviite  inthrmaii.>n  about  the  operational  and 

collection  capabilities  ofl  _ .  land  THINTHREAD.  We  determined 

that  NSA  provided  an  incomplete  i  HIN  tHKfcAL)  system  | 

Jstudv.  bin  wc  were  unable  to  confirm  which  version  of 
THINTHREAD  software  was  provided.  In  addition,  the  NSA  Chief  of  Signals 
Intelligence  stated  that  NSA  did  not  provide  any  technical  assistance  in  the  deployment 
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^study  The 


study  concluded 


as  stov^pipcd  arid  did  not  recommend  u  as  a  solution  for 


on  IheNSA  t- _ ,  - 

with  NSA’s  proposals 

containing  an  increase  ol  _ 

FY  2004  authorization  forgn  additional  L 
signals  processing  strategy  across  the  signal  intellig 


Congressional  staffs 
The  staffs  concurred 
FY  2003  House  bill 
processing,  and  in  the 
clop  and  support  a  long-term 
enterprise 


( U/iTO li Of  Audit  Response,  NSA  was  unable  to  provide  documentation  of  these  briefs 
to  the  audit  team,  documentation  01  subsequent  guidance  or  direction  from  congress,  or 
anv  other  information  that  documents  copgressional  dissatisfaction  with  THINTHRkAD 

«  % 

(Siaih  Management  Comments.  In-FY2003  and  FY2004.  during  budget  authorization 
and  appropriations  briefings  to  the  House  Permanent  Select  Comminee  on  Intelligence 
and  the  Senate  .Select.  Committed  on  Intelligence,  the  Congressional  staffs  recommended 
consolidating  all  processing  activities  into  a  single,  cohesive,  horizontally  integrated, 
evolutionary-development  procesamit  digital  network  exploitation  pro gnun  NSA 
responded  by  elevattngr  ""frs  the  overarching  digital  network 

exploitation  program,:  consolidating  processing  activities  within  one  organization, 
consolidating  systems  engineering  and  systems  engineering  management  and  control,  and 
strengthening  acquisition  management  capabilities.  ' 


(fcUM*  Audit 

network  qxj 
However. 

requirements 


umbrella  term  for  all  digital 
and  new  development 
to  process  the  digital  network  exploitation 


(KiOil)  Audit  Response.  Wc  agree  that  some  THrNTHREAI)  lessons  learned  were 
applied  lf__  Ibui  there  is  no  evidence  to  support  that  the 

TH1NTHR£aJl)  protect  continued  to  develop  and  support |  1  tor  the 

JPrognun.  The  THINTHREAD  deV«Wpffl«l  iAIldepi 


WPicn'Bir 

not  embodied  in  T 


_ _  TOpngfl!  MUdept  and 

y  OTnTTfldict  the  current  NSA  digital  network  exploitation  approach  and  is 
|  Internal  and  external  studies  show  that  attempts 
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Wm 


to  merge  [HtNTHRBAD  technology  into  the  architecture  based  on  4 


iccnnoiogy 

c*  were  hot  successful  or  technically  feasible  because  the  merger 
8  negative  PTfect  on  THINTHREAb  performance 


(44#J#t  Management  Comments.  C 


^system 


engineering 


develup  new  capabilities,  and 


♦ 

1  # , 

•# 

:  a  cursory  comparison  Between  Ihe  hko 

lftrlit/iK/  4 imnliM  tkn 

mg  to  Intelligence 


impact  of  Office  of  Inspector  General  Recommendation  A.  I  is  "3ev: 

Community  analyst  needs. 

H4  Audit  Response.  NS  A  management  continues  to  enhance 

capabilities,  even  (hough  its  systems  are  based  on  other  legacy  systems  and  legacy 

approaches  that  have  not  kept  up  with  today '%  technology  All  | 

capabilities  shown  in  Anne*  A  can  he addedto  the  existing  THiN  i  MKh.AU  archilee 

4 

(U/^SWBT Management  (  omments.  NSA  recommended  thai  the  audit  report 
emphasis  NSA's  cooperation  and  responsiveness  to  Congress 


TOie 


Aadit  Response.  Information  m  congressional  record*  notes  that  NSA  did  not 
explain  to  congress  why  THINTHREADdto  not  constitute  at  least  some  pan  of 
TRAILBLAZER  Instead,  NSA  delayed  deploying  TH INTHREAD.  Although  the 
THINTHREAD  deployment  study  showed  that  not  being  fully  documented  would  not 
have  a  negative  effect  on  mission  capability,  NSA  chose  to  document  THINTHREAD 
before  completing  its  deployment.  Most  ofTH IN  THREAD  deployments  could  have 
been  completed  phot  to  our  audit  The  THINTHREAD  deployment  plan  was  inadequate. 


(IV/POHO)  Management  Comment*.  NSA  recommended  that  the  audit  report 
emphasis  that  the  THINTHREAD  capability  is  valued  by  NSA. 


Audit  Response.  NSA  management  did  not  support  the  development  and  use  ot 
THINTHREAD  despite  the  opinion  of  many  NSA  cryptologic  experts  and  engineers  that 
TH  INTREAD  technologies  and  capability  were  viable  tools  for  deployment  for  digital 
network  exploitation 

CSl  Management  Comments.  NSA  recommended  that  the  audit  report  emphasis  that 
THINTHREAD  capability  is  integrated  within  the  NSA  processing  services 

(44  Audit  Response.  We  agree  that  some  lessons  learned  have  been  integrated  within 
the  NSA  processing  services,  however,  NSA  does  not  adequately  use  THINTHREAD 
and  its  capabilities. 
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l  Comments.  THfKIHREAD  does  not  possess  the  breadth  and  depth 
!  Intelligence  Cornmuju^bgcguse  iLs  ovc|  arching  program. 
Idocs.  Subordinating  lo  1  H INTHREAD 


V-  V.  loocs.  dUDonnnpiB _ _ 

uid  DC  Intelligence  C  ommunity  requirements 

'  •  •  %**/•• 

*  •  «**.*•  •  - _ 


rtWWJ  Autflt  Respwse:  IfNSA  added  the  services  lo  THINJHREAD  required  by  (he 
Intdligence  Cominuitjty.  THINTHREAD  would  have  the  breadth  and  depth  of  serv  ices 

o^iWhniploii  today's  |  | 


{fcUfili  NSA  continues  tot: 
legacy  approaches  instead 
used  immediately  NSA  n 
the  future  development  of 
known  at  NSAiand  the  di 


lion<!  of  dollars  enhancing  legacy  system  based  on 
(the THINTHREAD  interim  capability  that  can  be 
I  should  consider  THINTHRF.AD  as  an  option  for 
digital  network  exploitation  system  It  is  widely 
exploitation  community  that 


*  * - 

(9WR|)  Managepient  Comments. 

on  using  the  pesj  available  technol 
technologies,  is  dedicated  lo  w< 
NSA  corporate  vision  Migrating 
violate  this  strategy. 


IlUt  A  .strategy  for  collection  site  processing  is  based 
* '  which  allows  for  rapid  assimilation  of  evolving 
^ --Efluantglttk.fnd  »»  committed  to  the 

j.to  THINTHREAD  would 


fRMSUAndft 

NSA  Joint 


lit  Respon 

ii 


i»e.  The  most 

Team  state  that  ■  ica 

]  and  that  planned- ipihancements  will 
advancement"  The  evalm 


i  to  keep  up  with  technical  . 


not  provide  the  technical 
evaluations  also  state  that  the 


capabilities 

flawed  {therefore,  we  believe  that  this  condition 
shows  that  NSA  n  not  using  the  hesl  available  technology  nr  world-class  systems. 

, BUBmaimJ  -  -  ,t  I 

| _ J  In  addition.  NSA  comments  do  not  illustrate  how 

strategy. 


migrating  to  THINTHRKAD  would  violate  NSA 


Management  Comments.  NSA  accepted  thcl  Ii 
rhnology  Test  und  Evaluation. S'  and  Ihejl 

emitting  to  corporate  strategy  (TRAIL.nLA/  tK),  Mvam 


NCil 

Technology  ‘ 
committing  I 

management,  advancing  systems  engineering  and  systems  < 
incorporating  the  best  of  existing  processing  i 

dedicating  itself  to  condnua)  improvement  NSA'*  I _ ^ ^ ^ _ _ 

embodies  that  commitment.  It  is  based  on  state-of-the-art  technology,  which  maximally 
uses  commodity-based  hardware  and  industry  standard  engineering  and  development 


lies,  the 
\  report, 
i  and  acquistnon 
eenng  managancnl. 

i  THINTHREAD -and 

~ program 
cn  m« 
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<Ki  mcthp<|fl(log)C!r..to  allow  rapid  assimilation  of  new  technologies  and  rapid  adoption  of 
service  changes  while  at  the  same  time  preserving  a  stable  processing  environment 

ttttfu  Audit  Response.  NS  A  did  i 

study  that  i 
from  tbef" 

THIM 


^  •  Tjperatina  Svatem  into  the| 

of  the  team  statqd  that  technical  and  philosophical  diffi 
systems  lo;merge.  NfSA  officials  ituisted  that  they  try. 


t  recommendations  in  the  | _ 

I  including  the  operating  system 
;  reports  also  recommended  that 
because  it  could  provide  the  needed 

to  find  a  way  to  incorporate  theO 
rather  than  incorporate  the  Zone 
I  I  When  several  members 
osophical  differences  would  not  allow  the  two 


■ffWOfi  Management  Comments. 


NSA  stated  that  the  draft  report  omitted  critical 
vements  of  THINTHREAD  services 


lagei 

information  ibout  nevy  developments  and 

within  T  tt  NSAalac  ■ 

I  _ ^serviccSto  THINTHREAD  would  require  a  comprehensive  re- 


improi 

3  NSA  also  stated  that  any  attempt  to  migrate 


engineering  of  the  THftsTHRF-  .AD  framework  to  take  advantage  of  updates  to  signals 
intelligence  technology  and  to  moke  it  useable  within  the  existing  signals  intelligence 
system  The  cost  to  migrate  current  and  evolving  capabilities  while  maintaining  mission 
has  yet  to  be  quantified  \ 


(W#h  Audit  Response.  We  agree  that  some  lessohs  learned  from  THINTHREAD  have 
been  applied  tq  lsvstems.  but  NSA  did  not  document  and  the  audit 

team  did  not  see  evidence  ot  continuing  development  of  THINTHREAD.  In  Appendu 
D.  the  report  shows  that  all  of  the  services,  being  performed  by  f  ^  I 

U  and  that  the  estimated  cost  for 

In 


can 


he  report 
be  added 


to  the  THINTHREAD  system  i 


addition,  those  costs  do  not  include  the  substantial 
eliminating  the  operation  and  maintenance  cost  for 
THINTHREAD  has  its  own  databases,  provide 
and  does  not  use  the  NSA  traditional  | 


1  from 

i 

1  Also. 

i  own  graphics  interface  1 

for  analysts. 

approach. 


.Management  C  omments,  NSA  stated  that  the  draft  report  did  not  indicate  the  risks 
or  the  effects  dial  performing  any  level  of  change  would  have  on  the  existing  capability  of 
the  mission  NSA  also  slated  that  limitations  of  many  collection  sites  prohibit  additional 
systems,  would  exacerbate  problems  with  parallel  support  and  migration,  and  would  put  the 
mission  at  risk  The  total  nsk  of  migrating  capabilities  while  maintaining  mission  support 
has  not  yet  been  quantified 

UflMMf  Audit  Response.  The  DoD  IG  recommendations  that  NSA  deploy 
THINTHREAD  as  an  interim  solution  and  use  a  complementary  approach,  until 
THINTHREAD  is  fully  enhanced,  took,  into  account  the  risks  and  effect  that  might  have 
on  the  existing  capabilities.  Of  course.  NSA  must  identify  the  risks  and  effects  that 
changing  existing  capability  might  have  on  the  mission.  Collection  sites  limitations  was 
one  of  the  issues  considered  by  (he  original  THINTHREAD  developers.  NSA  cannot 
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tttaf  U  spending  millions  of  dollars  to  add  new  capabilities  to  systems  that  are  already 
operating.far  below  curTettl  mission  requirements  and  based  on  outdated  approaches  will 
not  fix  the  problems  of  the  NSA  digital  network  exploitation 


(G^Mnaagoment  Comments.  NSA  stated  that  the  audit  report  does  not  define  the 
meaning  of  “felly  enhanced  and  extended  THINTHREAD  *'  NSA  believes  that  a  “fully 
enhanced  and  ektenditi  THINTHREAD"  engineered  in  concert  with  the  NSA  corporate 
strategy  will  be  rpmarkahly  similar  to.  or  the  nmr  asl  I 

Alternatively,  if  this  “fully  enhanced  and  extended  THINTHREAD”  were  not  engineered 
according  to  the  NSA  corporate  strategy,  potentially  significant  changes  to  the  strategy 
would  cause  changes  to  subsystems  throughout  the  signal  intelligence  system  and  would 
interrupt  analysts  performing  thejr  national  security  duties  NSA  also  stated  that  the 
report  did  not  indicate  How  THINTHRF.AD  would  be  engineered  into  the  signal 
intelligence  system  or  itseffects  on  the  system.  The  NSA  and  the  Department  of  Defense 
Office  of  the  Inspector  General  did  imM  quantify  the  total  scope  and  impact  of  migrating 
current  capabilities  While  maintaining  mission  capabilities 


(Q  NSA  management  believes  that  THINTHREAD  does  not  have  the  potential  to  he  the 
sole  digital  network  exploitation  processing  system  THINTHREAD  provides  the 
greatest  value  to  the  signal  imrlWcnn--  system  and  to  the  analysts  by  remaining 

integrated  within  j _  1  1  * 

nsk  of  migrating  froml  .  Ito  I 


..  NSA  believes  that  the  scope,  cost  and 
THINTHREAD  are  not  characterized 


well  enough  to  supplant  existing,  corporate  signal  intelligence  strategy 


(^l  Audit  Response.  The  term  “fully  enhanced  and  extended  THINTHREAD”  was  used 
in  the  NSA  Modernization  Study  and  we  believe  that  it  means  that  NSA  will  enhance  and 
extend  THINTHREAD  to  provide  the  capabilities  needed  to  meet  mission  requirements 
The  final  system  would  be  "remarkably”  similar  to  i  1  however,  the 

new  TH  INTHREAD  would  be  able  to  fulfill  the  future,  as  well. as  the  existing,  input  and 
throughput  requirements.  I  1  study  and  the  Joint  Inspectors  General 

Inspection  Reports  state  that  the  technology  design  of  I  Its  flawed 

and  it  cannot  keep  pace  with  advances  in  technology  without  major  changes 


(^Management  C  omments.  The  cost  to  produce  “fully  enhanced  and  extended" 
THINTHREAD,  particularly  in  terms  of  level -of-efTori  and  skill  availability,  is  nol 
known  but  is  certainly  not  trivial. 


(0f  Audit  Response.  A  small  team  of  NSA  engineers  developed  THINTHREAD  in  less 
than  I  year  The  team  used  readily  available  commercial-off-the-shelf  software  for  RO 
percent  of  the  system  and  the  remaining  20  percent  was  developed.  Our  review  of  an 
NSA  employee  and  contractor  database  showed  that  several  members  of  the  onginal 
THINTHREAD  team,  still  work  in  some  capacity  at  the  NSA  and  are  available  to  start 
the  enhancement  of  THINTHREAD 


(0)  Management  Comments.  The  cost  of  modifying  the  rest  of  the  signal  intelligence 
system,  as  well  as  impact  on  other  parties'  systems,  to  conform  to  THINTHREAD 
operations  is  not  known  but  is  certainly  nol  tnvial. 

(dMsdil  Response.  To  continue  and  win  the  United  States’  fight  against  terrorism,  it  is 
imperative  that  NSA  deploys  the  best  technology  to  meet  mission  requirements 
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f*T  immediately.  NSA  must  determine  and  then  implement  the  agfii  solution  with 
proven  capability  The  failure  to  implement  the  right  solution  cuujd  result  in  loss  of 
intelligence,  which  could  lead  to  the  loss  of  American  lives  like  (hat  of  September 


II.  2001. 


{{^Management  Comments.  The  cost  to  support  multiple  platforms  with  a  migration 
without  affecting  the  mission  to  THINTHREAD  is  oot  known,  but.  is  certainly  not  trivial 

f  t  M  | 

Audit  Response.  After  THINTHREAD  is  hilly  enhanced  and  extended.  NSA  should 
determine  which  I  bvstems  miy  be  eliminated  Some  systems  and 

development  may  be  eliminated  now.  baaed  on  the  appropriate  analysis  and  review 
Eliminating  unnecessary  systems  would  result  in  a  substantial  reduction  in  operations  and 

maintenance  cost  for  NSA  digital  network  exploitation  systems. 

*  •  **  • 

Management  Comments.  The  -ac  h  ic  v  abt 1 1 ty, cost  and  effect  of  retraining  the 
intelligence  Community's  analysts  in  use  THINTHREAD  while  the  maintaining  mission 
is  not  known,  but  is  certainly  not  trivial . 


fa  Audit  Response.  Several  NSA  Intelligence  analysts  learned  to  use  THINTHREAD 
without  NSA  support;  they  stated  that  the  system  is  easy  to  learn  and  easy  to  use  A 
structured  training  course  could  only  enhance  the  analyst  training 


tf)>Atanagemcal  Comments.  The  resultant  THINTHREAD,  once  built  using  NSA’s 
corporate  strategy.  Is  not  likely  to  be  significantly  different  from  the  current  capability, 
which  is  embodied  within!  I 


(60 6*»  Audit  Response.  THINTHREAD  provides  significant  capabilities  that 

Idoes  not  and  cannot  provide,  such  as  processing  all  digital 


identifying  intelligence  data 


]  Such  capabilities  are  very  important  in 


m  Management  Comments.  NSA  stated  that  the  THINTHREAD  concept  of  operation 
is  different  from  that  used  by  today's  Intelligence  Community,  however.  NSA  supports 
THINTHREAD  components  successfully  integrated  into  the  signal  intelligence  system, 
complete  with  comprehensive  training  In  addition,  NSA  stated  that  it  started  to  create 
training  and  operation  materials  for  THINTHREAD  components  in  November  2002 
when  integration  activities  started  By  April  2003,  NSA  had  completed  courseware  for 
analyst  and  operator  training,  which  u  continually  updated  and  relied  on. 


>fik Audit  Response.  THINTHREAD  and  its  approach  is  a  radical  shift  from  NSA’* 
current  approach  to  digital  network  exploitation;  however,  it  presents  the  best  approach 
available  to  meet  existing  mission  requirements  and  should  be  considered  as  an  option  to 
meet  future  requirements.  NSA  management  has  not  accepted  the  THINTHREAD 
system  and  its  approach  as  the  solution  for  digital  network  exploitation  and  subsequently 
does  not  advocate  its  use.  As  a  result,  THINTHREAD  is  not  widely  used  within  NSA 
Analysts  who  do  use  THINTHREAD  either  trained  themselves  or  were  trained  by  other 
analysts  familiar  with  the  system.  None  of  the  analyst  we  interviewed  used  the 
courseware,  therefore,  we  are  not  commenting  on  the  adequacy  of  the  courseware  itself 
but  on  the  lack  of  knowledge  that  the  courseware  exists. 
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-{^Management  (  ommeots  TRAILBLAZER  welcomes  an  open  review  of 
rHINTH READ'S  2001  end-to-end  architecture  to  meet  the  requirements  of  the 
TRAlLBl-AZER- Technology  Demonstration  Platform  (Technology  Demonstration 
Platform)  because  wc  believe  that  NSA's  currenl  strategy  is  valid  and  will  be  revalidated 
("he  Program  Rxccutive  Office  (PEO)  is  taking  the  lead  in  establishing  an  independent 
review  team  for  performing  this  assessment,  with  the  intention  that  its  results  will  be 
completed  in  time  for  inclusion  into  the  Analysis  of  Alternatives  and  other  Milestone  B 
documentation  However,  the  performance  of  this  additional  study  will  have  an  impact 
on  both  the  cost  and  schedule  for  Milestone  B.  PEO  and  TRAILBLAZHR  will  estimate 
the  cost/schedule  impacts  after  receiving  estimates  from  the  review  team  and  will  decide 
at  that  time  whether  to  respond  with  a  Final  Concunencc  to  these  recommendations. 

Audit  Response.  The  recommendation  was  revised  to  reflect  that  THINTHREAD  is 
an  option  for  the  digital  network  exploitation  portion  of  the  Technology  Demonstration 
Platform 

■  Management  Comments.  NS  A  management  does  not  agree  with  the  Dof) 

Office  of  Inspector  General'*,  assessment  that  THINTHREAD  is  **a  better  technological 
and  more  affordable  solution  tor  digital  network  exploitation  than  the  solution  being 
developed  by  NSA.”  In  fact,  there  arc  significant  differences  between  THINTHREAD 
and  TRAILBLAZER's  missions. 


-ftWSI)  Audit  Response.  We  acknowledge  the  fact  that  the  current  scope  of  the 
TRAILBI -AZF.R  is  to  provide  the  information  technology  framewurk  for  the  entire 
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464  We  acknowledge  the  tael  that  the  initial  THINTHREAD  development  was  narrowly 
focused.  however.  THJfN/'THREAD  can  be  easily  upgraded.  NSA  is  being  disingenuous 
because  the  completed  studies  compared  only  the  front-end  of  THINTHREAD  and  Zone 
I  -  3  of  the  |  ~  ^  system  NSA  needs  to  conduct  a  study  to 

compare  entire  IHtlVIURhAD  sy**iernK>  current  digital  network  exploitation  systems. 


(0981)  Management  Commenu.  The  Technology  Demonstration  Platform  isa  prototype 
designed  (n  address  specific  program  risks and  to  satisfy  congressional  direction  for  <u. 


• 

• 

t 

« 

t  # 

• 

<£££!*  Audit  Response.  The 

framework  being  developed 

bvf 

Jis  based  on  the  existing^ 

development  and  subsequently  is  flawed  and  does  not  meet  current  mission  requirements 
and  will  not  meet  future  mission  requirements 


(0MSJJ  Management  Comments.  NSA  stated  that  THINTHREAD’s  end-to-end 
architecture  does  not  adequately  meet  the  operational  requirements  for  TRAIL-BLAZER  "j 
Technology  Demonstration  Platform  because  it  has  major  mismatches  or  deficits  in  all  key 
areas.  While  THINTHREAD  is  a  focused  operational  implementation,  the  Technology 
Demonstration  Platform  effort  is  focused  on  a  more  reusable  industry  standard  framework 
Thus,  the  Technology  Demonstration  Platform  framework  can  extend  operational  capabilities 
beyond  the  implementation  of  THINTHREAD  In  the  long  run.  the  Technology 
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Demonstration  Plalfoml  should  bt  dWc  10  integrate  T 
I  -*,-nU)  support  a  much  broader  set  of  Digital  Network  Information  requirements  than 
-  icd  by  THINTHREAD 

y.v\  •  . 

f£i>  Audit  Response.  This  is  an  inaccurate  statement  THINTHREAD  was  built  using 
readily  available  Commercial  off-the-shelf  technology  and.  when  fully  extended  and 
enhanped,  aftfuld  ncK.bc  difficult  to  integrate  with  Technolog)'  Demonstration  Platform 
effort i(.  however,  ti -is  <Hir  understanding  the  1  cchncitngv  Demonstration  Platform  is  hemu 
ate  with  lh<J  | 


tevelopment  efforts  which  is  the  maior  concern  An  external  team  or 
I  he  used  jo  determine  what  parts  of  the  Technology  Demonstration 
Platform  devetapmeht  should  continue  while  THINTHREAD  and  its  approach  and 
technologies  are  considered  as  an  alternative  lo  the  system  being  developed  by 

i  •  -  • 

* • ^  •  «  V 

(U  Management  Comments.  NS  A  stated  that  the  Technology  Demonstration  Platform 
will  provide  a  commercial,  extensible,  and  service-based  “plug  and  play"  framework 
Though  THINTHREAD  does  employ  some  commercial  -off-the-shelf  development 
products  and  technologies,  that  employment  is  it  a  lower  level;  overall.  THINTHREAD 
resides  on  a  custom-deVeloped  framework .  \ 

Audit  Response.  THINTHREAD  was  built  using  readily  available  commercial  off- 
the-shelf  technology  and  resides  on  a  custom-developed  frame  work  that  is  inexpensive, 
small,  and  scalable.  Contrary  to  the  statement  above,  the  THINTHREAD  framework  is 
extensible  and  is  service-bas«id  plug  and  play'. 


Management  Comments.  NS  A  slated  that  the  Ti 
includes  major  development  efforts  to  transform  f 


ion  Platform 


Transformation  of  NSA  databases  was  nol  a  I  HlNTKKEAn  Scfivify  I  his  is  jus!  one  a  * 

major  difference  between  Technology  Demonstration  Platform  and  THINTHREAD 

t  • 

9  • 

ffcfflit)  Audit  Response.  These  statements  are  inaccurate  Although  THINTHREAD 
development  started  in  early  1999  and ’.was  operational  in  late  1999,  it  is  still  state-of-the- 
art  and  revolutionary  in  ilesnai  THINTHREAD  nrovutes  itai  I 


Malf  NSA  documented  its  problems  with  its  current  assortment  of  databases  These 
databases  hold  large  amounts  of  data  with  little  or  no  intelligence  value,  r 
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|t>v  lHtWHKEAD  was  still  up  and  riinninqjnd^  dtfectefl  that  it 
immediately  be  stopped  and  personnel  involved  be  removed  and  |  ,/• 

system  take  over  the  requirement  .•  I 


(SiMni 


t  C'oi 


tu  NSA  slated  that  the.li 


.A’ 
•  A* 


‘«K«  ■ 

has  a  significant  information  Assurance  effort. 


intormation  assurance  issues  i 

deployed  and  certified. 


klaRU 


ion  Platform 


J 


to 


TSS  nut  address 
THfNTHRF.AD  could  be 


Audit  Response  Original  THIN  THREAD  developers  maintained  that  information 
assurance  could  be  handled  by  another  software  application,  this  lit  routinely  done 
throughout  the  Federal  Government  and  DoD 


Audit  Response.  NSA  statements  are  inaccurate  Anyncccssary  commercial-off- 
the-shelf  tools  and  legacy  tool  could  be  integrated  with  THfNTHREAD  The  signal 
intelligence  data  produced  by  JHINTHREAD  is  much  easier  for  analysts  to  retrieve  and 
nalYM  thm  tattling  Urals  aqd  capabilities  emphasized  by-NSA  management  Although 

has  been  completed  for  THlNTHRF.AD,  we  believe  that 
experts  available  at  MSA  and  within  local  area 
t  problems  The  THINTHREAD  design  and  approach 


c 


number  o! 

companies  will  avoid  an1 
considers  information  [* 


J 


] 


(£?)  Management  C  onuneott.  NSA  managements  stated  that  TRAILS  LAZER  and  the 
Technology  Demonstration  Platform  have  invested  considerable  effort  to  build  a  signal 
intelligence  architecture  that  is  open  and  scaleablc.  applies  DoD.  commercial,  and  NSA 
standards;  and  that  is  documented  in  DoD  Architecture  Framework  views 
THINTHREAD  architecture  and  acquisition  process  were  not  consistent  with  the  guide 
lines  of  DoD  Directive  5000.2,  and  the  program  exhibited  shortcomings  that  constituted 
“a  serious  risk  to  successful  deployment.''  according  to  the  March  25. 2002,  Technology, 
Test,  and  Evaluation  Assessment  mentioned  on  pages  20-21  of  the  Office  of  Inspector 
General  Report  and  quoted  extensively  in  the  Office  of  Inspector  General's  Appotdix  G. 
This  is  especially  critical  documentation,  given  the  emphasis  on  Horizontal  Integration 
and  the  DoD  Global  Grid  evolution 


<<p> Audit  Response.  NSA  comments  are  inaccurate  llir  THINTHREAD  architecture 
is  open  and  scaJeablc  Information  that  we  reviewed  and  previous  Inspector  General 
reports  show  that  the  NSA  acquisition  processes  is  not  always  consistent  with  the 
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J024  (i) 


OMlWT^iOiQI  18.1 


Tb4- guidelines  df  DoD  Directive  5000  2.  The  program  shortcoming!!  referenced  in  the 
Technology,  Test,  and  Evaluation  Assessment  dealt  mainly  with  program  documentation, 
which  the  report  noted  did  not  present  a  significant  deployment  risk. 

— (So'Bli.Maaageroent  Comment!.  NSA  management  comments  stated  that  the  Executive 
Summary’s  assertion  that  NSA  “disregarded  solutions  to  urftcnl  national  |g  >mtv 
is  nol  accurate.  TH INTHREAD  was  deployed  tq 


Audit  Response.  NSA  comments  arc  inaccurate.  Maiur  issues  raised  in  the 
management  comments  are  addressed  in  the  report,  specifically  the  issue  of 
TH1NTHREAD  scalability  In  the  FY200I  Intelligence  Authorization  Bill  (CMS  1-23- 
2001 ),  December  17.  2001.  the  House  Permanent  Select  Committee  on  Intelligence 
directed  that  |  |of  the  NSA  budget  be  used  to  immediately  denim 

THINTHREAD  to  sites  selected  ten  iplimal  response  to  I  land  tor 

1.  nic  nsa  bnefed  thh  rouse  ivwiafuw  vice; 
utmnttnecon  iiucinggnwwm  utinthread  would  be  deployed  iof~~hiic*.  wui  the 
THINTHREAD  Project  Management  Office  modified  this  lol  kites  maddition.  all 
services  that  are  needed  to  exploit  the  digital  network  cap  be  aBoed  to  THINTHREAD 
For  example,  the  THINTHREAD  approach  to  digital  nttwork  exploitation  is  a  shift  from 
existing  practices  at  NSA  because  | 
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-ItTMuigeintnl  Comments.  NS  A  management  stated  that  the  J3n[)  Inspector  General 
Report's  citation  of  the  NTA  Study's  recommendation  to  support  adoption  of 
TUfNTHREAD  also  needs  clarification  The  Study  did  not  reuommend  THIN  THREAD 
as  a  InniHcnn  solution.  but  rather  as  an  interim  capability  (frptntype  alternative  to  the 


^VhAudit  Response.  NSA  comments  are  inaccurate.  The  NSA  Modernization  Study 
recommended  that  THINTHRFAD  be  included  as  an  alternative  solution  to 
TRAIL-BLAZER 

/  • 

•4€TManagement  Comments  NSA  management  comments  stated  (hat  although  the 
Executive  Summary  indicated  that  TRAIL3L  AZER  excluded  THINTHREAD  from 
consideration  as  an  alternative  framework-  It  should  he  nointad  out  that  documentation  on 
both  THINTHREAD  and  the|  .*  |was  provided  to  bidder*  for 

the  Concept  Development  PH33Z  i  nis  tact  i*  aocumemea  in  the  December  2001  Follow 
Up  Report  to  the  NTA’s  NSA  Modernization  Study  (p  13)  However,  none  of  the 
competing  contractors  chose  to  yse  THINTHREAD  in  their  proposals,  which  is  why  this 
system  was  not  included  originally  in  the  Analysts  of  Alternatives. 

Audit  Response.  The-Director.  Signal  Intelligence  Directorate,  stated  that 
information  about  THINTHREAD  technologies  and  approach  was  placed  in  the  bidder's 
library.  Placing  files  0out  THINTHREAD  tn  a  huge  repository  of  other  files  and 
making  those  files  accessible  to  contractors  does  not  adequately  support  the  NSA 
Modemizauon  Study  recommendation  to  include  THINTHREAD  in  the  altemative-ot- 
analysis  process  for  TRAILBLAZER  In  addition,  information  provided  to  the 
contractors  for  <hc  Concept  Development  Phase  by  NSA  directed  them  to  ensure  their 
proposals  for  system  met  the  current  NSA  architecture  and  framework,  to  include 

_Jervices.  and  other  legacy  constraints,  which  THINTHREAD 
approach  does  nol  comply 

(U/.TO  UOjJManageroent  C  omments.  NSA  management  comments  stated  that  the 
TRAILBLAZER  hut  long-established  plans  fbi  external  assessment  of  the  Technology 
Demonstration  Platform  for  preparation  for  Milestone  B  The  Independent  Verification 
and  Validation  (IV&V)  of  29  Evaluation  Criteria  is  being  conducted  by  NSA's 
independent  Office  of  Corporate  Assessments'  Test  and  Technical  Evaluation  element 
(TTE)  The  Early  Operational  Assessment  and  Operational  Assessment  will  be 
performed  by  JITC,  which  had  established  a  presence  in  TRAILBLAZER  spaces  by 
November  2003  Results  of  both  activities  will  be  provided  to  the  Milestone  Decision 
Authority  before  Milestone  B 

(U//fW$Qi  Audit  Response.  TRAILBLAZER  postponed  Milestone  B  scheduled  for 
December  2004  because  of  technical  issues  with  the  Technology  Demonstration 
Platform.  February  2005  is  the  earliest  date  that  TRAILBLAZER  will  be  ready  for 
MILESTONE  B 

(UTTFOUQk Management  Comments.  NSA  management  comments  stated  that  the 
results  of  the  independenl  assessment  of  THINTHREAD  and  TRAILBLAZER 
Technology  Demonstration  Platform  will  be  provided  to  the  MDA  by  Milestone  B. 
currently  scheduled  for  December  2004 

(9S/9H  Management  Comments.  NSA  management  comments  stated  that  the  Executive 
Summary's  statement  that  TRAIL-BLAZER  development  “wasted  hundreds  of  millions 
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of  dollars"  is  not  accurate.  As  the  NT  A  Modernization  Study  observed. 
rRAlLBLAZER  is  not  only  about  digital  network  exploitation  hm  also 
cultural  transformation  (people,  processes,  and  policies)  andl 
I  "  1  And.  ^though  TRAILBLAZER  is  currently  in  the 

developmental  phase,  it  has  already  delivered  significant  capabilities 


(BiWHfr  Audit  Response.  The  NSA  management  comments  are  inaccurate.  The  report 
Executive  Summary  does  not  claim  that  TRAILBLAZER  development  “wasted  hundreds 
of  millions  of  dollars.''  Instead,  this  phrase  is  only  used  to  accurately  describe  the 
specific  hotline  complaint  allegation  that  initiated  this  audit 
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Management  Comments.  NSA  management  comments  suited  that  the  ti 

- - aB=^  ‘  tel  lc 


costs  for  the  prototype  were  closer  to( _ 

in  the  Office  of  Inspector  General  Report. 


^rather  than  the  | 


Audit  Response,  the  total  costs  of  the  Initial  Transformation  Activities  and 
|  ’  (accurately  reflect  information  received  from  the  NSA 

Wga  WTTTgg  ana  qn  m  JHClUfle  cost  for  NSA  employees- 

•  *  * 

Management  Comments.  NSA  management  comments  also  questions  the 
Executive  Summary’s  statement  that  "NSA  modified  or  suppressed  studies  and 
comparisons  that  favored  THINTHREAD  over  TRAILBLAZER."  and  stated  that  the 
audit  report  did  not  identify  where  the  information  was  obtained. 

% 

(K)  Audit  Response.  We  havp  documented  information  to  support  this  statement, 
however,  because  of  fear  of  reprisal,  we  agreed  to  keep  the  sources  anonymous 

The  Office  of  Inspector  General  Report's  citations  of 
UNTHREAD  Technical  Review  (February  2001 )  and 
(READ  Follow-up  for  D/DIR  (March  2001) 
were  also  incomplete  and  contained  inaccuracies.  The  presentation  of  the  results  of  the 
February  review  on  page  1 3  of  the  Office  of  Inspector  General  Report  was  only  a  subset 
of  the  total  recommendations  and  findings  For  example,  although  this  study 
(commissioned  by  the  Chief  of  the  Mission  Applications  Group,  not  the  D/DIR  as  stated 
in  the  Office  of  Inspector  General  Report)  recommended  adoption  of  THINTHREAD ’& 


4.  Provide  requested  resources  to  the  THINTHREAD  team  (dollars  and 
people). 

5 .  (MU)  Provide  TH INTHRbAD  proper  management  support. 
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MSA  (bXD  14(c),  (bX3)  50  USC  3605,  (bX3)  50  USC 
J024  (i) 

MSA  (b)(1 )  1  4(c),  (b)(3)  50  USC  3605 
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6  (UkU  In  light  of  this,  not  all  of  the 
required. 


-T*-~ 


} 


lysiem  is 


*  ***•••( 

-t€f  Audit  Rttpoau:  Our  wording  reflects  information  received  from  our  sources  We 
reviewed  all  of  the  comments  below  and  checked  our  sources  and  made  adjustments  as 
required 


4.  I&MU^SA  never  provided  the  requested  rcsouacs  (dollars  and  people)  to 
the  TH INTHREAD  team.  Instead.  NSA  dismantled  the  team  ana  did  little  to 
enhance  THINTHREAD  ' 

5  iSMifeThere  was  ne  evidence' that  NSA  provided  proper  management  support 
lo  enhance  THINTHREAD. 

_ a _ a 


(SMSll  Management  Comments.  NSA  management  comments  stated;  that  the  follow-up 
technical  review  (March  2001 )’,  requested  by  Deputy  Director,  also  stated  that  both  the 
THINTHREAD  and  TRA1L&LA7.ER  approaches  were  of  value  It  recommended 


ZZhr 

r  General  Rt 


m  flataa  rm  mnrn  inwMiaamB  wrirtwtir 

of  Inspector  General  Report's  descriptions  of  the  I  kind  Technology,  Test,  and 

Evaluation  Studies,  although  the  Appendices  are  far  more  accurate  than  the  references 
contained  in  the  body  of  the  Report 


mg  om« 


THINTHREAD  lot: 

management  tq  make 


ic  March  2001  technical  review*  recommended  that  the 
system  be  developed  and  deployed  to[ 


.  .  _ ]t 

]  The  study  team  was  pressured  by  N  S  A 


and 


recommendations  for  the  development  and  deployment  of  the 
|ystcm 
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Appendix  K.  Report  Distribution  (U) 

(U) 

Office  of  the  Secretary  of  Defense 

Linda  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics 
Linda  Secretary  of  Defense  (Comptroller) 

Linda  Secretary  of  Defense  for  Intelligence 

Assistant  Secretary  of  Defense  for  Networks  and  Information  Integration 


Other  Defense  Organization 

Director,  National  Security  Agency 
Inspector  General,  National  Security  Agency 
Inspector  General,  Defense  Intelligence  Agency 
Inspector  General,  National  Geospatial-Intelligence  Agency 
Inspector  Genera),  National  Reconnaissance  Office 


Congressional  Committees  and  Subcommittees,  Chairman  and 
Ranking  Minority  Member 

Senate  Subcommittee  on  Defense,  Committee  on  Appropriations 

Senate  Committee  on  Armed  Services 

Senate  Select  Committee  on  intelligence 

House  Subcommittee  on  Defense,  Committee  on  Appropriations 

House  Committee  on  Aimed  Services 

House  Permanent  Select  Committee  on  Intelligence 
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NATIONAL  SLCURiTY  agency 
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11  February  2005 


memorandum  for  deputy  assistant  inspector  general  for 

INTELLIGENCE  AUDITS 

SUMMIT  il)//F(lUU41oI) OHi  Audit  Report  Requirements  for  the  TRAILBLAZER 
and  TH I  NTH  READ  Systems.  dated  December  15,  2004  <Re|»ori  05-INTEL- 
0:i,  Project  No  D2003AL-OI00)  INF<  lRMATIt IN  MEMORANDUM 


(1 1//TVH  MHIn  response  to  the  Onl)  O|0  rrquetl  for  additional  information  on  the 
TRAlLBIAZER  and  THINTHREAD  re|iort.  NRA/CSS  provide*  consolidated  corporate 
comments  from  our  Signals  Intelligence  Lhrectot  ate,  Acquisition  Organization,  and 
Informal  ion  Technology  Infrastructure  Services  Organization  (now  the  Information 
Technology  Directorate). 

NKA/C-SS  Response 

^^The  Program  Executive  Officer  <PEO>  for  NSA  Transformational  Programs  is 
committed  to  funding  the  timely  completion  nf  the  requested  assessment  of 
TH1NTHREAD  applicability  to  the  TRAILBLAZER  Technology  Demonstration 
Platform  and  ensuring  the  results  of  the  assessment  ore  considered  in  the  preparation 
and  conduct  of  TRAILBLAZER  Milestone  B  iMS  R>.  Consistent  with  duties  and 
rtfh|H)nsil#ilities  of  the  PEO.  the  independence  uf  the  ;iKS4n»sinHiit  is  assured.  The 
.iLjumiif  i;iL.m>s^  of  uicornuniiintf  TH I  NTH  READ  as  an  option  for  meeting  the 

1  1  requirements  for  TRAILBLAZER  TDP  and  subsequent 

incorporation  into  the  TRAILBLAZER  Analysis  of  Alternatives  will  he  determined  by 
the  PRO  nt  the  completion  of  the  assessment 

1  OWJu bsei]Uik(it  to  the  DoD  OIG  report  the  TRAILBLAZER  Milestone  has  been 
scheduled  for Ilate  tn  the  fourth  quarter  of  FY05  from  the  previously  scheduled  earl ier 
MS  B  Accordingly,  thi>  now  allows  sufficient  tinv<  fur  the  conduct  nf  the  assessment 
recommended;  by  the  DoD  OlO  ,As  costs  for  the  mwvsm'  nt  will  hr  covered  hy  the  PEO. 
and  due?  to  th£  revised  schedule,  risk  to  the  TRAILBLAZER  program  is  minimal. 
Thttrcforr,  a  Risk  Mitigation  Plan  is  not  appropriate  The  assessment  will  be  finalized 
by  Mnv  2005  •  ; 


If  you  hove  any  questions  nr  need  additional  information,  please  rontnrt 
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<  mmi  ii)i'!f«‘i  i.m. 


-jctiemrci  <kssinc»tn»< 


NSA  STAFF  PROCESSING  FORM 


PURPOSE 


l  U//TOI.MJ  >  To  provide  the  recommended  PEO  input  to  the  consolidated  NSA  response  regarding 
actions  requested  by  the  '‘final’’  Department  of  Defense  ( DoD)  ( )flfice  of  Inspectin’  General  i  OK i ) 
report  06-INTEL-03.  Requirements  for  TRAJLBLAZER  and  Tl  UNTHREAD  Systems  of  15 
December  2004 

BACKGROUND  /  DISCUSSION: 

< U//R>LN*>  In  their  June  ‘04  Draft  of  the  Audit  Report,  the  DoD  OIG  made  the  following 
recommendation.  "Designate  a  team  to  asses*  the  ability  of  THiNTHREAD  or  its  technologies 
as  a  long-term  solution  to  meet  the  operational  requirements  of  the  TRAILBLAZER  Technology 
Demonstration  Platform." 

-e0t  In  response  to  this  draft,  in  August,  the  Agency  concurred  "with  comments."  The  following 
is  an  excerpt  from  those  comments;  "  The  Program  Executive  Office  i  PEO)  is  taking  the  load  in 
establishing  an  independent  review  tenm  for  performing  this  assessment,  with  the  intention 
that  its  results  will  lx-  completed  in  time  for  inclusion  into  the  Analysis  of  Alternatives  and 
other  Milestone  B  documentation." 

t«TAt  the  time  of  this  statement  (Aug  *04).  the  T RAJ  1 . B LAZER  Milestone  B  (MSB)  was 
scheduled  for  December  ‘04.  When  the  final  report  was  published,  the  PEO  Assessment  had  not 
yet  begun.  Concerned  that  the  assessment  could  not  ho  completed  in  tune  to  support  a 
December  milestone,  the  DoD  OIG  requested  via  the  final  report,  that  they  be  provided:  “NSA 
plans  to  eliminate  the  cost  and  schedule  risks  for  completing  the  assessment." 

-it1**  Since  release  of  the  final  Audit  Report,  a  DIRNSA  -Chartered  Transformational  Advisory 
Panel  (TAP)  recommended  THAI  LB  LAZER  he  rebaselined.  The  MSB  is  now  planned  for  late 
Fourth  Quarter  FY0f». 
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i  (  '/TTrt^WaJliu  of  this  response  Jim*  been  (‘uprdinated  wiin  Lnii 

rh*  TRAII.BLAZER  Program  Management  Office  I  I 

»  ♦ , 

RECOMMENDATION 

1  >  [ '//POI  !<*-  A  formal  Risk  Mitigation  Plan  is  not  recommended  The  PEO  its 
committed  tt»  conducting  the  asseKsment  The  team  is  currently  being  i.irinud— a_ 
kick-piT  mlg  is  planned  for  March.  and  the  team  ir  i-xp«-cted  to  report  to  Mr  I 


NLT  May  lllut.  The  cost  ancJ  schedule  impact  of  conducting  the  .iwesHHitnl 
pose  vorv  low  risk  lo  tile  Transformational  Programs. 


2.  •  I  WPt  lUtJu  Recommend  the  response,  provided  as  an  attachment,  he  forwarder!  to 
the  Dul)  OIG  as  part  of  the  Agency's  consolidated  response  to  the  Final  Report. 
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(U)  The  Office  of  the  Deputy  Inspector  Cicncral  for  Intelligence  of  the 
Department  of  Defense,  intelligence- Audit  prepared  this  report  Personnel  of  the 
Office  of  the  Inspector  General  of  the  Department  of  Defense  who  contrihulod  in 
the  repon  are  listed  below 
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